Why Likelihood Inference Can Look More Certain Than It Is
Selection-model likelihoods can be difficult to optimize and may contain weakly identified parameter combinations. As a result, software can return a coefficient, standard error, and test statistic even when the numerical fit or the assumptions behind those quantities are inadequate. The key discipline is to distinguish a returned number from a supported inference.
This tutorial focuses on three errors with direct consequences: treating a fit that has not converged as a maximum, interpreting standard errors from a singular or nearly singular Hessian as trustworthy, and treating a flat profile likelihood as precise evidence. These errors are related, but they are not interchangeable. A failed optimizer is a numerical warning; a singular Hessian is a curvature and identifiability warning; a flat profile is evidence that the data provide limited discrimination among values of the target parameter.
For example, if subject \(i\) has covariates \(x_i\), observed outcomes \(y_{i,\mathrm{obs}}\), and response pattern \(r_i\), a selection model specifies an outcome distribution and a response probability conditional on the full outcome. Its observed-data contribution can be written as
The integral is over the missing outcome components. The likelihood must represent the specified selection model and the observed data; optimization cannot repair a misspecified likelihood. The usual likelihood calculations discussed below also rely on appropriate regularity conditions. In particular, ordinary Hessian-based standard errors require a well-defined interior estimate and locally identifiable parameters. Standard likelihood-ratio cutoffs require their own regularity conditions; boundary estimates or nonstandard calibration can invalidate a routine chi-square reference distribution.
Error 1: Calling an Unfinished Optimization a Maximum
An optimizer’s convergence flag is useful, but neither a “success” message nor a printed parameter vector proves that a global maximum has been found. Conversely, a nonconvergence message means the reported vector has not been certified as a maximum by that optimization run. A run may stop because it reached an iteration limit, encountered numerical problems, or could not satisfy its stopping rule. The likelihood may still increase nearby, or a different starting value may lead to a better solution.
The relevant objective is the observed log likelihood on the same data, under the same parameterization and numerical settings. A candidate that has a larger log likelihood than another candidate is better by that objective, but the comparison alone does not prove either is a global maximum. Check the optimizer status alongside the score or gradient, the local curvature, the parameter constraints, and the stability of likelihood evaluations. In selection models, numerical integration accuracy also matters: a likelihood that changes materially when quadrature is refined cannot support fine-grained optimization conclusions.
Worked Example: A Stopped Run Is Not a Verified Maximum
Suppose a hypothetical selection-model analysis reports a candidate with log likelihood \(\ell=-412.8\). The optimizer stopped at its maximum iteration count and reported a gradient with components \(0.08\) and \(-0.03\). A second start, using the same model and data, reaches \(\ell=-411.9\), also without a convergence certificate.
Compare the objective values. Since \(-411.9>-412.8\), the second candidate has the larger log likelihood. The difference is
Thus the first candidate cannot be treated as the best fit found in these runs. For orientation only, the likelihood at the first candidate relative to the second is \(\exp(-0.9)\approx0.407\). This is not a likelihood-ratio test: neither candidate is established as the required maximum, and the difference is not being calibrated against a reference distribution.
Check the stopping evidence. The iteration-limit message says the stated stopping rule was not met. The nonzero gradient components are additional reason to investigate, although their importance depends on parameter scaling. A rescaling that makes a gradient look numerically small does not, by itself, establish a maximum.
Conclusion. Neither run is yet a verified maximum. The next step is to diagnose scaling and numerical integration, continue or restart optimization as appropriate, and compare additional candidate solutions. Until the fit is adequately supported, likelihood-based standard errors and tests computed at either candidate should not be presented as established inference.
Error 2: Treating a Singular Hessian as a Standard-Error Calculator
At an interior maximum, the observed information matrix is \(I(\hat{\theta})=-\nabla^2\ell(\hat{\theta})\), the negative Hessian of the observed log likelihood. Under regularity and adequate local identification, the inverse information approximates the covariance matrix of the maximum-likelihood estimator. A Wald standard error for a parameter is the square root of its corresponding diagonal variance estimate.
This calculation depends on invertible, appropriately positive curvature. If the information matrix is singular, its inverse does not exist. If it is nearly singular, inversion can magnify small numerical errors and produce very large or unstable variances. A software package may still print standard errors through generalized inverses, regularization, or rounding; a printed value is not evidence that the ordinary inverse-information approximation is valid.
The curvature justification follows from a local Taylor expansion. Near a genuine interior maximum, the score is approximately zero, and the log likelihood is approximately quadratic:
When the information has small eigenvalues, the quadratic drop is small along the corresponding directions. The likelihood then changes little as parameters move in those directions, so their estimates are poorly determined by local curvature. Inverting the information turns a small eigenvalue into a large variance component: if an eigenvalue is \(a>0\), its inverse is \(1/a\). If \(a=0\), no finite inverse exists. This is why a singular-Hessian warning is not a minor formatting issue.
Worked Example: Near-Singularity Makes a Wald Error Unstable
Suppose the observed information for two hypothetical model coefficients, each measured in its model’s coefficient units, is
The eigenvectors are proportional to \((1,1)\) and \((1,-1)\). Their eigenvalues are \(8.0000\) and \(0.0008\), respectively: adding the diagonal and off-diagonal elements gives \(8.0000\), while subtracting them gives \(0.0008\). The inverse information therefore has eigenvalues
The standard deviation along the weakly curved direction is approximately \(\sqrt{1250}=35.36\) coefficient units. By contrast, a small change in the entries of \(I\), potentially arising from rounding or numerical error, could substantially change its smallest eigenvalue and therefore this standard error.
Conclusion. The matrix is technically invertible, but its near-singularity makes a Wald summary highly sensitive. The large value is a warning about weak information and numerical instability, not a precise measure of uncertainty to accept without investigation. Inspect eigenvalues and eigenvectors, confirm the likelihood and optimization are stable, and consider likelihood-based or bootstrap inference where their assumptions are justified.
Error 3: Reading a Flat Profile as Precise Evidence
For a scalar target parameter \(\psi\), let \(\lambda\) denote nuisance parameters. The profile log likelihood is the best log likelihood attainable at each fixed value of \(\psi\):
A profile that changes little over a wide range of \(\psi\) means that the observed data do not strongly distinguish those values after nuisance parameters are allowed to adjust. It does not mean that the parameter is known precisely. A profile-likelihood interval is often formed from values satisfying \(2\{\ell(\hat{\psi})-\ell_p(\psi)\}\) below a chosen cutoff. The cutoff and its calibration must be appropriate to the model and inferential setting; a routine chi-square cutoff is not automatically valid for boundary or nonstandard cases.
A flat profile can arise from weak identification, parameter trade-offs, limited data information, or a combination of these. It can also be falsely flat if the nuisance-parameter optimizations failed at fixed values of \(\psi\). Thus the shape is informative only when the profile has been computed reliably. A broad profile is a reason to report limited precision, not to treat the maximizer’s location as decisive.
Worked Example: A Broad Profile Does Not Pin Down the Treatment Coefficient
In a hypothetical selection-model analysis, the target \(\psi\) is a treatment coefficient on the log-odds scale. The maximum profile log likelihood is \(-210.0\) at \(\hat{\psi}=0.2\). At \(\psi=-0.8\), the profile log likelihood is \(-210.7\); at \(\psi=1.2\), it is \(-210.9\). Assume for this illustration that the fits at these fixed values have converged and that the ordinary 95% profile-likelihood cutoff of \(3.84\) is justified.
At the lower value, the likelihood-ratio statistic is
At the upper value, it is
Both values are below \(3.84\), so both remain inside the stated profile-likelihood acceptance set. The displayed values alone do not establish the interval’s endpoints, but they do show that values spanning two log-odds units are not excluded by this cutoff. The likelihood ratios relative to the maximum are \(\exp(-0.7)\approx0.497\) and \(\exp(-0.9)\approx0.407\), respectively.
Conclusion. These results do not provide precise evidence that \(\psi=0.2\). They indicate that the likelihood is comparatively tolerant of substantially different treatment coefficients over the displayed range. The full profile and its calibration are needed for interval construction; a narrow Wald interval based on unstable curvature would not erase the weak information shown here.
A Practical Diagnostic Sequence
The following sequence helps separate numerical failure from inferential uncertainty. It is not a guarantee of validity: the selection model, observed-data likelihood, and calibration assumptions still need to be appropriate.
Confirm that the objective is the observed-data likelihood for the intended selection model and data, including the required integration over missing outcomes.
Inspect the optimizer’s stopping status, score, constraints, and likelihood evaluations. Compare sensible starting values and verify that integration accuracy is adequate.
Check whether the estimate is interior and whether the observed information is positive definite and well-conditioned. A singular or near-singular matrix undermines ordinary Wald inference.
Verify nuisance optimization at each fixed target value. Examine the profile’s width and shape, then use a cutoff with justified calibration.
Use Wald summaries only when their local quadratic approximation is credible. Profile likelihood can better reveal asymmetry or flatness; likelihood-ratio tests and parametric bootstrap procedures also require appropriate assumptions and reliable fits.
How the Main Approaches Differ
| Approach | What it summarizes | Important limitation in an unstable fit |
|---|---|---|
| Wald inference | A local estimate and standard error from inverse observed information | Depends on reliable curvature, a valid interior estimate, and a useful local quadratic approximation |
| Profile likelihood | Likelihood support for target values after optimizing nuisance parameters | Can be misleading if those optimizations fail; flat profiles honestly indicate weak information |
| Likelihood-ratio inference | The likelihood difference between constrained and unconstrained fits | Requires both fits to be adequate and a justified reference distribution or calibration |
| Parametric bootstrap | The sampling behavior generated under a specified fitted model | Cannot rescue a poorly fitted or inappropriate model; results depend on the generating assumptions |
These methods answer related but distinct questions. Profile likelihood is often useful for seeing asymmetry and weak support that a single standard error can conceal. It is not automatically reliable merely because it avoids a Wald calculation. Likelihood-ratio inference likewise depends on valid constrained and unconstrained fits, and its calibration may be nonstandard. A parametric bootstrap can reflect finite-sample behavior under a specified model, but it inherits the model’s assumptions and any instability in fitting it.
Common Mistakes and Limits
- Wrong: “The software returned coefficients, so the fit is a maximum.” Correction: A parameter vector is an output, not proof of optimization. Check stopping evidence, score, competing starts, and the likelihood computation.
- Wrong: “The standard error is finite, so the Hessian is adequate.” Correction: Near-singular matrices can yield finite but unstable values. Inspect conditioning and weak-curvature directions, not just whether a number printed.
- Wrong: “The profile is flat, so the estimate is very precise because it barely changes.” Correction: A flat profile means many target values have similar likelihood support; it signals limited precision.
- Wrong: “A profile-likelihood interval automatically solves every inference problem.” Correction: Its nuisance fits must be reliable and its cutoff appropriately calibrated, especially near boundaries or under nonstandard conditions.
No diagnostic by itself establishes that a selection model is correctly specified or that its parameters are identified by the available data. Numerical checks assess computation; curvature and profile checks assess local or target-specific information; calibration checks assess the reference distribution. These are complementary checks, not substitutes for one another. When they conflict, the conflict is itself a result to investigate rather than a reason to select whichever output looks most reassuring.
Check Your Understanding
Use the distinctions among optimization, curvature, and profile information to answer each question.
- A run stops at its iteration limit, and a second run obtains a higher log likelihood under the same model and data. Can the first run be called the verified maximum? Explain.
- What does a zero eigenvalue of the observed information imply for the ordinary inverse-information covariance estimate?
- A target value has profile log likelihood \(0.6\) units below the maximum. Calculate its likelihood-ratio statistic and explain what a flat profile would imply.
- Why does a profile-likelihood interval not automatically have valid coverage in a selection model with a boundary estimate?
Solutions. (1) No. The first run failed its stopping criterion and has a lower objective value than another candidate, so it is not the best fit found, let alone a verified maximum. (2) The information matrix is singular, so its ordinary inverse does not exist; the usual inverse-information covariance estimate is undefined. (3) The statistic is \(2(0.6)=1.2\). Whether that value is inside an interval depends on the justified cutoff, but a profile that remains close to its maximum across a broad range indicates weak discrimination among those target values, not precision. (4) Boundary estimates can violate the regularity conditions behind routine likelihood-ratio calibration. The profile may still be useful, but its cutoff or calibration must be justified for that setting.