Two Different Questions: Did the Fit Stop? Does the Model Fit?
An NLME program can report successful termination while the model describes the data poorly. Conversely, an iteration-limit warning can appear even when the objective has barely changed and the fitted predictions look reasonable. These are not contradictions: an optimizer reports on a computational process, whereas model adequacy concerns whether the model’s assumptions and predictions are reasonable for the data and scientific question.
A message such as “converged” usually means that one or more software-specific stopping criteria were met. It does not establish that the optimizer found the global optimum, that the estimates are uniquely determined by the data, or that the chosen model is adequate. A message such as “maximum iterations reached” means that the algorithm stopped because it used its allowed iterations; it does not, by itself, show that the estimates are unusable.
The central practical task is therefore to assess three kinds of evidence separately: the stability of the objective function, the plausibility and stability of estimated parameters, and the model’s predictions in relation to the observations. None of these checks alone settles every question, but together they provide a more informative assessment than the termination message alone.
Model and Meaning of the Objective
For orientation, consider observations \(y_{ij}\) for individual \(i\) at occasion or time \(j\). A standard NLME model represents them as
Here \(f\) is the structural model, \(\theta\) denotes population parameters, \(b_i\) denotes individual random effects, and \(\epsilon_{ij}\) denotes residual error. The exact residual distribution and the way random effects enter the model must be specified for a fitted likelihood to have its intended interpretation. The objective function is a numerical summary that the estimation procedure seeks to optimize; for example, it may be minus twice a log-likelihood or an approximation to one.
A numerical comparison of objective values is meaningful only when the same data, model, objective definition, and relevant numerical settings are used. For a likelihood-based objective that is minimized, a lower value indicates a better optimized likelihood for that same setup. It does not say whether the model is scientifically adequate. Objectives from different model structures may also include different terms or approximations, so their values should not be compared casually.
Objective Stability and What a Stopping Rule Means
An optimizer typically updates an estimate repeatedly. Let \(Q^{(k)}\) be the objective at iteration \(k\). One simple way to describe the change between iterations is
The absolute change \(\Delta Q^{(k)}\) records direction as well as size; the relative change \(r^{(k)}\) describes the change on a scale adjusted for the objective’s magnitude. This relative measure is a useful explanation of objective stability, but it is not a universal NLME convergence criterion. Software may use different criteria, and an objective can be stable while parameters are still moving along a shallow region of the likelihood.
The justification for examining relative change is straightforward: a change of 0.01 is small compared with an objective near 1,000, but large compared with one near 0.02. Dividing the absolute change by the previous objective’s magnitude makes these cases distinguishable. The \(\max(1,|Q^{(k-1)}|)\) denominator avoids division by a value near zero. Even a very small relative change only says that the objective changed little over that step; it does not prove that the current point is a global optimum or that the model fits well.
Worked Example: Stable Objective at Termination
Suppose a hypothetical fit reports an objective decreasing from 512.84 to 512.21 and then to 512.18 over its final iterations. The optimizer’s documented objective-change criterion is \(r^{(k)}<0.0001\). Assess the last change.
The last absolute change is \(512.18-512.21=-0.03\). Its relative magnitude is
Since \(0.0000586<0.0001\), this change satisfies that stated objective-change criterion. The objective also decreased, as expected for this minimization step. This supports the limited conclusion that the objective was stable at the end according to this particular criterion. It does not establish that other stopping conditions were met, that the estimates are plausible, or that the model’s predictions adequately represent the observations.
Check Estimates and Predictions, Not Just Messages
Parameter plausibility is a contextual check. Confirm that units are consistent, signs and scales are scientifically interpretable, and estimates imply sensible behavior over the observed time range. “Plausible” is not a substitute for a formal statistical assessment: a plausible estimate can still be poorly identified, and an unexpected estimate may reflect either model-data conflict or a numerical problem.
Prediction checks ask a different question: do fitted predictions capture relevant features of the data? Residuals, defined here as observed minus predicted values, can reveal systematic discrepancies. A trend in residuals over time, concentration, or another meaningful covariate may indicate that the structural or residual model is missing a feature of the data. The interpretation also depends on which predictions are used. Individual predictions that incorporate estimated individual effects can fit the data more closely than population predictions; they should not be treated as interchangeable evidence.
Worked Example: Plausible Parameters but a Prediction Pattern
In a hypothetical one-compartment oral PK fit, suppose the estimated clearance is \(CL=5.0\) L/h and volume is \(V=20.0\) L. At four post-dose times, population predictions and observed concentrations are as shown. Assess the parameter scale and the residual pattern. Concentrations are in mg/L.
| Time (h) | Predicted (mg/L) | Observed (mg/L) | Observed − predicted (mg/L) |
|---|---|---|---|
| 1 | 10.0 | 11.0 | 1.0 |
| 2 | 8.0 | 8.5 | 0.5 |
| 4 | 5.0 | 3.5 | −1.5 |
| 8 | 3.0 | 0.5 | −2.5 |
The estimated elimination rate constant implied by these parameters is \(CL/V=5.0/20.0=0.25\) h\(^{-1}\). Under a one-compartment model with first-order elimination, the corresponding half-life is
The units cancel to give hours, and the parameter values imply a positive, finite half-life. This arithmetic check does not establish that the values are appropriate for a particular drug or population; that judgment needs scientific context. The residuals, however, change from positive at early times to increasingly negative at later times. Thus the predictions are too low at the first two times and too high at the later times, with a pattern rather than random scatter around zero.
The root mean square residual for these four points is \(\sqrt{(1.0^2+0.5^2+(-1.5)^2+(-2.5)^2)/4} =\sqrt{9.75/4} \approx1.56\) mg/L. That single summary does not encode the time pattern: the table is important. The hypothetical fit might have terminated successfully, but this residual pattern warrants investigating whether the model’s predictions are adequate for the intended use.
Reading Maximum-Iteration Warnings and Repeated Claims
A maximum-iteration warning should trigger a review rather than an automatic rejection or acceptance. Check how the objective changed near the limit, whether parameter estimates continued to move materially, and whether the reported estimates and predictions are usable for the intended purpose. A stable objective with stable, plausible estimates is more reassuring than a steadily changing objective, but neither pattern overrides problematic predictions or unsupported assumptions.
Repeated messages that a model “converged” require the same separation of questions. Repeated fits with similar objective values and similar estimates provide evidence of reproducibility for those runs under those settings. Repeated convergence messages alone do not show that the model is adequate. If objective values are similar but estimates differ appreciably, the data may not distinguish those parameter combinations well; similar objective values can also occur when different estimates imply similar predictions. Record what was repeated and compare the numerical results, rather than counting messages as independent confirmations.
Worked Example: A Warning and Repeated Convergence Claims
A hypothetical fit reaches its limit of 500 iterations with a maximum-iteration warning. Its objective values over the final three recorded iterations are 620.70, 620.45, and 620.40. Two subsequent runs report successful termination, with objective values 620.38 and 620.42. Their estimated clearances are 5.1 and 6.8 L/h, respectively. What can be concluded?
For the last recorded change before the warning, \(\Delta Q=620.40-620.45=-0.05\), and \(r=0.05/620.45\approx0.0000806\). The final objective change is small relative to the objective scale, and the two later objective values differ by only \(0.04\). These figures are evidence that the objective values are similar across the reported fits. They do not show that the first run met the software’s stopping rule, because it explicitly ended at the iteration limit.
The clearance estimates differ by \(6.8-5.1=1.7\) L/h, or \(1.7/5.1\approx33.3\%\) relative to the first estimate. That difference is substantial enough to investigate rather than dismiss on the basis of similar objectives. Compare the corresponding predictions and other estimates, verify the runs used the same data, model, and numerical settings, and consider whether the estimates are sufficiently determined for the scientific purpose. The successful-termination messages from the later runs are relevant computational evidence; they do not erase the first warning or establish model adequacy.
A Practical Review and Common Pitfall
A disciplined review keeps the computational and scientific questions distinct:
Record whether the run met its stated stopping conditions, stopped at a limit, or reported another estimation failure. Do not translate every message into a binary “good fit” or “bad fit.”
Inspect the objective near termination and compare runs only when data, model, objective definition, and relevant numerical settings match. Stability supports a computational conclusion, not a conclusion about adequacy.
Verify units and scientific interpretability, then inspect predictions and residual patterns relevant to the intended use. Investigate systematic discrepancies even if termination was successful.
Report the message, objective behavior, parameter checks, and prediction findings separately. If evidence is incomplete or conflicting, say so rather than declaring the model validated.
A common mistake is: “The software says converged, therefore the model fits the data.” The reasoning fails because the stopping rule concerns the optimizer’s numerical progress, not whether the model’s assumptions describe the data. The correction is to treat convergence as one piece of computational evidence and assess parameter plausibility and predictions independently.
The opposite mistake is to reject every fit with a maximum-iteration warning. The warning does signal that the specified stopping condition was not reached, but the objective, estimates, and predictions still contain diagnostic information. Report the warning, assess that evidence cautiously, and do not present the fit as an unqualified successful convergence. These checks do not replace more specialized troubleshooting when a specific numerical failure is present.
The distinction matters in practice because a model used for simulation, dose selection, or prediction needs more than a reassuring software message. Conversely, dismissing a potentially informative fit solely because of one warning can conceal a model whose results are numerically stable enough to investigate further. The appropriate conclusion is proportionate to the evidence actually checked.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A minimization objective changes from 400.0 to 399.8. Calculate the relative change using the definition in this tutorial. What does it establish?
- A run reports successful termination, but residuals are positive early and negative late. Is the model necessarily adequate? Explain.
- Two runs report convergence and have nearly identical objective values, but their parameter estimates differ. Name two checks to make before interpreting the estimates.
- Why should objective values from different model structures not automatically be compared as if they had the same interpretation?
Solutions. (1) The relative change is \(|399.8-400.0|/400.0=0.2/400.0=0.0005\). It quantifies a small change on this scale; whether it meets a stopping rule depends on that rule, and it establishes neither global optimality nor adequacy. (2) No. The trend suggests systematic prediction error and should be investigated despite successful termination. (3) Check that the runs used the same data, model, and numerical settings; compare implied predictions and assess whether estimates are stable and scientifically interpretable. (4) Different structures may use different objective terms or approximations, so the numerical values may not be directly comparable; the objective definition and fitting setup must be understood first.