Why Estimation Settings and Convergence Evidence Belong in a Report
An NLME fit is obtained by applying a numerical procedure to a specified model and dataset. A statement such as “the model converged” is too vague to show what was computed or what evidence supports that conclusion. Readers need to know which estimation method was used, which settings controlled the calculation, how the software terminated, and which checks were examined.
This reporting concerns the computation for a fixed model and analysis dataset. It does not replace the final model specification or the record of how candidate models were chosen. The central distinction is that termination status is a software outcome, whereas convergence evidence is a set of numerical observations supporting the claim that the algorithm reached a stable solution. Neither, by itself, demonstrates that the model is scientifically adequate.
What to Report About the Computation
Begin by naming the estimation method as specifically as the implementation allows. For example, distinguish a first-order conditional estimation method from a first-order conditional method with interaction, or state that a Laplace approximation or stochastic approximation expectation-maximization method was used. These methods do not generally calculate the same numerical approximation in the same way. If relevant to the method, state the approximation, integration, or simulation settings that affect the calculation.
Also identify the software and version, the optimizer or estimation engine when it is separately selectable, and the important numerical controls. Depending on the method and implementation, these may include maximum iterations, convergence tolerances, quadrature points, Monte Carlo sample size, random seed, parameter scaling, bounds, and initial estimates. Report settings that materially determine the computation; do not imply that a setting was used if the software does not expose it or it was not recorded. A software default is still a setting: name it as a default only when that is known and verifiable.
A useful report distinguishes the requested settings from what actually happened. State whether the algorithm reached a stated stopping criterion, exhausted a limit, stopped after an error, or was interrupted. Include material warnings, such as failed likelihood evaluations or a non-positive-definite covariance estimate. If a run was repeated from different initial estimates, distinguish those runs and say which result was retained and why. The exact objective convention matters too: identify whether values are reported as the log likelihood, negative log likelihood, or minus twice the log likelihood, and ensure that comparisons use the same convention and data.
How to Read Common Convergence Checks
Many optimizers seek a local minimum of an objective function, often a negative log likelihood or a related approximation. Near a smooth interior minimum, the objective should change little under small parameter updates, and its gradient should be close to zero. A gradient is the vector of first derivatives of the objective with respect to the estimated parameters. Because parameters may have different units and scales, a scaled gradient can be more informative than unscaled components. Report the software’s definition of a scaled gradient if it is needed to interpret a threshold.
A small objective change alone is weak evidence: an algorithm can make tiny steps because it is stalled, at a boundary, or numerically unstable. A small parameter change alone has the same limitation. The checks are more informative together, interpreted alongside the termination message and warnings. Some implementations use different stopping rules, scaling conventions, and tolerances, so there is no universal cutoff that can be applied without qualification.
Where an observed-information or Hessian matrix is available, its local curvature can be checked. For a smooth objective being minimized, a positive-definite Hessian at an interior stationary point supports the interpretation of a strict local minimum. It does not establish that the solution is the global minimum. A failed curvature check can also have several causes, including a flat or poorly determined direction, a boundary estimate, or numerical error; it should be reported rather than concealed.
Here \(Q\) is the objective being minimized and \(\hat{\theta}\) denotes the estimated parameter vector. This condition is a local stationarity condition, not a test of model correctness. In practice, software evaluates finite-precision approximations and applies its own criteria, so a report should give the observed diagnostic and the criterion used rather than claim exact equality.
A Short Justification for Using More Than One Check
Suppose an algorithm stops because successive objective values differ by less than a tolerance. That condition constrains the change in objective between the last two iterations, but it does not directly constrain the gradient. A flat region can have small objective changes even when the current point is not sufficiently close to a stationary point. Conversely, in a steep region, a small parameter step can still leave a notable gradient if the step was curtailed by a constraint or numerical issue. Thus, distinct diagnostics check distinct features of the iteration.
The logic can be summarized as follows: the termination rule describes why the software stopped; the objective and parameter changes describe the size of recent updates; the gradient addresses local stationarity; and curvature checks address the local shape around the estimate. Agreement among these checks strengthens the numerical account, but cannot prove that no better solution exists elsewhere.
Name the estimation method, relevant approximation, software, and version.
Report the tolerances, limits, integration or simulation settings, and starting strategy that apply to the run.
Give the termination status, objective convention and value, and any warnings or failed runs.
Report relevant diagnostics against their criteria, and avoid treating them as proof of global optimality or model adequacy.
Worked Example: Reporting a Deterministic Estimation Run
Consider a hypothetical NLME analysis using a fixed dataset and model. The software output records a Laplace approximation, a specified quasi-Newton optimizer, a maximum of 1,000 iterations, and a gradient tolerance of 0.001. The run terminates with “converged,” uses 146 iterations, and reports a final minus-two log likelihood of 812.46. Its maximum scaled gradient is 0.0008. No likelihood-evaluation warnings are recorded.
The observed gradient is below the stated tolerance because \(0.0008 < 0.001\). The run also stopped before reaching its iteration limit, and the output contains no recorded likelihood-evaluation warning. Those facts support reporting that the software met its stated convergence criterion. They do not establish global optimality: the result is still a solution found by a numerical search from its chosen starting values.
A concise report could read: “The model was estimated using the Laplace approximation with [software and version] and its quasi-Newton optimizer. The maximum iteration limit was 1,000 and the scaled-gradient tolerance was 0.001. The run terminated with the software’s ‘converged’ status after 146 iterations; the final minus-two log likelihood was 812.46 and the maximum scaled gradient was 0.0008. No likelihood-evaluation warnings were recorded.” The bracketed software detail must be replaced with the actual recorded name and version, not guessed.
Worked Example: Interpreting Multiple Stopping Diagnostics
In a second hypothetical run, the reported stopping criteria are an absolute objective change below 0.001 and a maximum scaled parameter change below 0.001. At the last iteration, the objective changes from 430.2170 to 430.2167, and the maximum scaled parameter change is 0.0007. The maximum scaled gradient is 0.0042; the software reports a gradient tolerance of 0.001.
The absolute objective change is \(430.2170 - 430.2167 = 0.0003\), which is below 0.001. The parameter-change criterion also passes because \(0.0007 < 0.001\). However, the gradient criterion does not pass: \(0.0042 > 0.001\). Therefore, it would be inaccurate to report that all convergence diagnostics met their stated tolerances. The report should state which stopping rule ended the run and disclose that the gradient exceeded its criterion. The small objective and parameter changes do not cancel this discrepancy.
This result warrants follow-up according to the analysis protocol and software behavior—for example, checking scaling, numerical evaluations, and whether an alternative starting point gives a stable result. The report should describe what was actually done, not retroactively label the fit “converged” because some diagnostics were small.
Worked Example: Comparing Runs From Different Starting Values
Suppose a hypothetical analysis is run from three starting values using the same method and objective convention, minus twice the log likelihood. Run A terminates with a convergence status at 502.31. Run B terminates with a convergence status at 502.30. Run C reaches its maximum of 800 iterations at 509.80 and has a scaled gradient of 0.012, above the stated 0.001 tolerance.
Runs A and B have similar objective values: \(502.31 - 502.30 = 0.01\). This agreement is useful evidence that these starting values reached similar solutions. Run C did not meet the gradient criterion and its larger objective value is not comparable evidence of a converged solution. A transparent account would report all three outcomes, state that A or B was retained under the prespecified rule, and give the reason for that rule. Similar values from two starts increase confidence in numerical stability over those starts; they do not prove that a lower objective cannot be found from another start.
This comparison is interpretable only because the runs used the same data, model, objective definition, and estimation settings apart from their starting values. If the estimation method or approximation settings differ, the objective values may not represent directly comparable calculations.
Diagnostics, Alternatives, and Limitations
The relevant evidence depends on the estimation method. For a deterministic optimizer, iteration counts, objective changes, parameter changes, gradients, boundary behavior, and curvature checks may be available. For simulation-based methods, the report should additionally describe applicable simulation settings, such as the number of draws, random seed, and any method-specific stabilization controls. If repeated runs vary because of simulation, report that variability rather than presenting a single run as if it were deterministic. Do not substitute a diagnostic from one method for a criterion that the implementation does not provide.
Alternative numerical strategies can help investigate a problematic fit: changing starting values, using a different optimizer, increasing integration accuracy, or using another supported estimation method. These are not interchangeable settings to be silently adjusted until a preferred result appears. Document any change, its rationale, and the result. Comparisons are meaningful only when the objective and data are sufficiently comparable; an approximation change can change the numerical target or its accuracy.
Convergence evidence has important limits. A locally stable estimate may still be affected by weak identifiability, model misspecification, data problems, or an approximation that is inaccurate for the problem. Those questions require their own assessments. A non-positive-definite covariance matrix is a material warning, but it does not by itself identify one unique cause. Likewise, a software message saying “converged” should not be treated as a statistical test, a guarantee of valid standard errors, or confirmation that the model describes the data adequately.
Common Reporting Mistakes
- Writing only “the model converged.” State the method, settings, termination status, and relevant observed diagnostics.
- Equating a successful message with a global optimum. The algorithm’s status supports only the stated numerical stopping claim; it does not rule out other solutions.
- Reporting a tolerance without the observed value. Give both, with the direction of comparison, such as a maximum scaled gradient of 0.0008 against a 0.001 tolerance.
- Hiding failed or boundary runs. Disclose material warnings, limit-exhausted runs, and boundary estimates when relevant to the reported computation.
- Comparing unlike objective values. Identify the objective convention and avoid direct comparison when data, model, approximation, or objective definitions differ.
- Calling convergence evidence a model check. Numerical stability and model adequacy are different questions; report them separately.
Check Your Understanding
Answer each question, then compare with the solutions.
- A run has an objective-change tolerance of 0.002 and an observed change of 0.001. What can be concluded from this diagnostic, and what cannot?
- A software run reports “converged,” but its maximum scaled gradient is 0.006 against a stated tolerance of 0.001. How should this be described?
- Two runs have minus-two log likelihoods of 610.4 and 610.3. Name two conditions that should hold before treating these values as a useful comparison.
- Why does a positive-definite Hessian at an estimated interior solution not establish that the solution is globally optimal?
Solutions. (1) The objective change is below tolerance because \(0.001 < 0.002\), so that particular criterion is met. It does not establish that the gradient criterion, other stopping rules, or model checks are met. (2) Report the software status and the gradient discrepancy: \(0.006 > 0.001\), so the gradient criterion was not satisfied; do not claim that all convergence checks passed. (3) The runs should use the same data and objective convention, and should have the same model and comparable estimation method/settings. (4) Positive definiteness supports a strict local minimum for a smooth objective at that point. It describes local curvature only and does not rule out a lower objective elsewhere.