Why stopping rules need interpretation
A joint model may estimate longitudinal fixed effects, random-effects covariance parameters, survival-model coefficients, and parameters linking the longitudinal and event processes. An iterative algorithm must decide when to stop updating this collection. A stopping rule makes that decision from numerical quantities; it does not certify that the model is identifiable, that the selected solution is the global maximum, or that numerical integration is sufficiently accurate.
This tutorial compares three common criteria: change in parameter estimates, change in the objective function, and gradient size. It then considers a practical sensitivity check: refitting the same model with tighter stopping tolerances and examining whether the estimates change meaningfully. The goal is not to choose one universal threshold, but to understand what evidence each rule provides and what remains unchecked.
The likelihood here is the objective actually used by the fitting algorithm. In a joint model it may be evaluated approximately, for example using numerical integration. Tightening an optimizer's stopping tolerance does not by itself improve that integration. Keep the model, data, likelihood calculation, and integration settings fixed when isolating the effect of a stopping tolerance.
Three common criteria
A parameter-change criterion stops when successive estimates are close. A direct version is \(\max_j|\theta_j^{(k+1)}-\theta_j^{(k)}|\leq\varepsilon\). This can be unsuitable when parameters use different scales or units: a change of 0.001 may be negligible for one parameter but important for another. A scale-aware alternative defines, for each component \(j\), a positive scale \(s_j\) and checks \(\max_j|\theta_j^{(k+1)}-\theta_j^{(k)}|/s_j\leq\varepsilon\). Scales can be chosen to reflect meaningful parameter magnitudes; software may instead use absolute and relative tolerances. The scale convention and threshold should be reported or checked in the software documentation.
A likelihood-change criterion stops when successive objective values are close. One common form is \(|\ell(\theta^{(k+1)})-\ell(\theta^{(k)})|\leq\varepsilon_{\ell,\mathrm{abs}}+\varepsilon_{\ell,\mathrm{rel}}\max(1,|\ell(\theta^{(k)})|)\). Here the absolute and relative thresholds set the permitted change on the objective's scale. A small increment says that the objective is barely changing from one iteration to the next; it does not necessarily say that the parameters are close to a unique optimum.
A gradient criterion checks the score, the vector of first derivatives of the log-likelihood. At an unconstrained, differentiable interior maximum, the score is zero. Because score components have parameter-specific units, it is useful to inspect a scaled norm rather than an unscaled maximum. For example, with a diagonal matrix \(D\) containing parameter scales, one can monitor \(\|D\nabla\ell(\theta)\|_\infty/\max(1,|\ell(\theta)|)\), where the infinity norm is the largest absolute component. This is a dimensionless scaled score; its threshold is still a numerical choice, not proof of a maximum.
| Criterion | What a small value indicates | What it can miss |
|---|---|---|
| Parameter change | Little movement between consecutive iterates under the chosen scaling | Slow movement, poor scaling, or a flat ridge along which estimates remain uncertain |
| Likelihood change | Little objective improvement between consecutive iterates | Material parameter movement on a flat surface, or an iterate far from a stationary point |
| Gradient size | Small first-order slope under the chosen scaling | A saddle point, a local rather than global maximum, or an inaccurate gradient |
For parameters at boundaries, such as a variance constrained to be nonnegative, the unconstrained score need not be zero at a constrained maximum. Boundary-aware procedures instead assess feasible directions or appropriate Karush–Kuhn–Tucker conditions. A very small unconstrained gradient is therefore not the only possible signature of a valid constrained solution.
Why the criteria can disagree
The criteria answer different questions. Parameter change concerns the sequence of estimates; likelihood change concerns the sequence of objective values; gradient size concerns the local slope at an estimate. A flat likelihood surface can make the objective change very little even while parameters move appreciably. Conversely, an algorithm may take tiny steps because of damping or numerical safeguards while the gradient remains appreciable. For these reasons, a robust practical assessment often considers more than one criterion and checks the fitted estimates under tighter tolerances.
This approximation is local and is not a convergence test by itself. It explains why a likelihood criterion can be uninformative about parameter movement when the surface is flat. Earlier material on likelihood profiles, ridges, and information-matrix eigenvalues addresses related signs of weak identification; a stopping rule cannot remove those features of the model and data.
Worked Example: A Scaled Parameter-Change Rule
Suppose a hypothetical joint-model iteration updates an association parameter from 0.40 to 0.43 and a log residual standard deviation from \(-0.20\) to \(-0.18\). Use scales \(s_1=0.10\) and \(s_2=0.20\), and a tolerance of 0.25 in the scaled maximum-change rule.
The scaled changes are \(|0.43-0.40|/0.10=0.03/0.10=0.30\) for the association parameter and \(|-0.18-(-0.20)|/0.20=0.02/0.20=0.10\) for the log residual standard deviation. Their maximum is 0.30. Since \(0.30>0.25\), this iteration does not meet the specified parameter-change rule.
The unscaled changes, 0.03 and 0.02, might look small, but their relative importance depends on the chosen scales. The calculation supports a decision about this particular rule only; it does not establish that either estimate is statistically precise or that the fit is at a maximum.
Worked Example: Small Likelihood Change, Nontrivial Parameter Movement
Consider hypothetical successive iterates for an association parameter and a longitudinal slope: \((0.60, 1.20)\) and \((0.68, 1.16)\). Their log-likelihoods are \(-812.431\) and \(-812.428\). Suppose the likelihood-change tolerance is 0.005 on the absolute change in log-likelihood.
The change in log-likelihood is \(|-812.428-(-812.431)|=0.003\), and \(0.003<0.005\). The likelihood-change rule is met. However, the association estimate changed by \(0.68-0.60=0.08\), while the slope changed by \(1.16-1.20=-0.04\). If those shifts matter for interpretation or downstream predictions, the small objective increment is not enough to call the estimates stable.
This pattern is compatible with a relatively flat region of the objective, but the two iterations alone do not prove that explanation. A tighter-tolerance refit and other diagnostics are needed to assess whether the parameter values settle down.
Worked Example: Checking the Gradient Rule
At a hypothetical unconstrained interior estimate, suppose the scaled gradient components are \((0.0004,-0.0021,0.0007)\). Use the infinity norm criterion with threshold 0.001.
The scaled gradient norm is \(\max(0.0004,0.0021,0.0007)=0.0021\). Since \(0.0021>0.001\), the gradient rule is not met. The largest component corresponds to the second scaled parameter direction. It signals a remaining first-order slope in that direction, subject to the accuracy of the gradient calculation.
If the estimate were on a parameter boundary, this unconstrained calculation would need a boundary-aware interpretation. Even for an interior estimate, passing the gradient rule would show only that the scaled first-order slope is small; it would not distinguish a maximum from a saddle point or establish global optimality.
Testing sensitivity to tighter tolerances
A practical check is to refit the same model with progressively tighter stopping tolerances. Keep the data, model specification, starting value, integration settings, and other numerical controls fixed as far as possible. Compare key parameters on their own scales, not just the final log-likelihood. If several runs use different starting values, they test a different question as well; do not attribute every difference solely to tolerance.
Save parameter estimates, the objective value, the reported stopping criterion, and any relevant numerical settings.
Reduce the specified parameter, likelihood, or gradient threshold while holding other settings fixed. If practical, repeat with a further tightening.
For each key parameter, calculate the difference between the tighter and baseline estimates. Also consider scientifically relevant predictions or contrasts.
Stable values across these runs support numerical stability with respect to the tested stopping tolerances. They do not establish model adequacy, identifiability, or integration accuracy.
Worked Example: Quantifying Tolerance Sensitivity
A hypothetical joint-model fit is run twice with identical data, model, starting values, and integration settings. The baseline run uses a stopping tolerance of \(10^{-4}\); the tighter run uses \(10^{-7}\). The estimates are as follows.
| Parameter | Baseline | Tighter tolerance | Absolute difference |
|---|---|---|---|
| Association parameter | 0.72 | 0.724 | 0.004 |
| Longitudinal slope (units/year) | 0.180 | 0.179 | 0.001 |
| Log residual standard deviation | -0.350 | -0.347 | 0.003 |
For the association parameter, the absolute change is \(|0.724-0.72|=0.004\). Relative to the baseline magnitude, this is \(0.004/0.72\approx0.00556\), or about 0.556%. The slope changes by \(|0.179-0.180|=0.001\) units per year. The log residual standard deviation changes by \(|-0.347-(-0.350)|=0.003\).
These values suggest that the estimates are not very sensitive to the tested tightening, given the precision shown. Whether a difference of 0.004 in the association parameter matters depends on the scientific scale and use of that parameter. The comparison does not show that the estimates are unbiased or that quadrature error is negligible, because integration settings were held fixed.
Diagnostics, alternatives, and limitations
A useful assessment includes the actual criterion value and its threshold, not only a software label such as “converged.” Inspect whether estimates are still changing over recent iterations, whether likelihood values behave as expected for the algorithm, and whether a gradient calculation is credible. Finite-difference gradients can be sensitive to step size and numerical noise. A gradient check at the reported estimate, considered together with curvature information, can help assess the local numerical solution, but it is not a substitute for tolerance sensitivity.
Parameter-change and likelihood-change rules are common alternatives when an algorithm does not provide a reliable gradient, as can occur in some EM procedures. A gradient criterion is natural for gradient-based optimization but depends on derivative scaling and accuracy. Using multiple criteria can be more informative than relying on one, although requiring all criteria to pass does not fix a misspecified model or a poorly identified parameter. Likewise, a stable estimate can be stably wrong if the model or numerical approximation is inadequate.
In joint models, parameters may have different units, and association parameters may be weakly informed by the available longitudinal visits or event follow-up. A tolerance sensitivity check should therefore focus on scientifically important parameters and quantities derived from them, not merely on whether the optimizer prints a success message. If estimates change under tighter tolerances, the direction and size of the change are informative; they do not by themselves identify the cause.
Common pitfall
Wrong reasoning: “The log-likelihood changed by less than the tolerance, so the parameter estimates must have converged.” A small objective increment only establishes a small change in the objective under the chosen rule. It does not bound parameter movement, especially on a flat ridge.
Correction: Check parameter changes on sensible scales, inspect the gradient when available, and compare estimates from tighter-tolerance runs. Treat these as complementary numerical checks. Do not confuse numerical stability with statistical precision: uncertainty estimates and identification require separate evaluation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A likelihood-change rule has tolerance 0.01. The absolute change between the last two objective values is 0.008, while a key parameter changes by 0.06. Has the likelihood rule passed, and what does that establish?
- Why should parameter changes often be scaled when a model contains parameters with different units?
- A fitted interior estimate has a scaled gradient infinity norm of 0.0006 with threshold 0.001. What does passing this rule establish, and what does it not establish?
- In a tolerance sensitivity check, optimizer settings are tightened and estimates barely change, but integration settings are held fixed. What aspect of numerical reliability has been assessed, and what has not?
Solutions. (1) Yes: \(0.008<0.01\), so the stated likelihood rule passes. This establishes only that the objective changed by less than its threshold; it does not guarantee that the parameter changed little or is stable. (2) A raw change has different practical meaning on different scales, so scaling makes the comparison more interpretable and prevents a large-unit parameter from dominating solely because of its units. (3) The scaled gradient rule passes because \(0.0006<0.001\); it indicates a small scaled first-order slope under the calculation used, but does not prove a maximum, global optimality, or parameter precision. (4) The comparison assesses sensitivity to the tested stopping tolerances. It does not assess whether the fixed integration approximation is accurate.