Diagnosing Local Maxima in Mixture Estimation

Why Local Maxima Need to Be Diagnosed

Latent class joint models are fitted by maximizing an observed-data likelihood that combines class-specific contributions. That likelihood is generally not a simple surface: different starting values can lead an optimizer to different stationary solutions. A program can report convergence for a solution that is not the best one found, or can stop at a point that has not actually converged. Repeating a fit and comparing its results is therefore part of checking the computation, not just a way to obtain more output.

A key distinction is between repeated fits that reach the same solution and fits that reach different solutions. In a latent class model, the class labels can be permuted without changing the model's likelihood. Thus, different-looking parameter tables do not necessarily represent different solutions. Conversely, equal or nearly equal likelihoods do not by themselves establish that two parameter estimates are the same.

Likelihood and Conditions for a Meaningful Comparison

For independent subjects \(i=1,\ldots,n\), let \(Y_i\) denote subject \(i\)'s observed longitudinal and event data, and let \(C_i\) be the latent class. Write \(\pi_{ig}(\theta)\) for the probability of class \(g\) given the model's specified covariates, and \(f_{ig}(Y_i\mid\theta)\) for the class-specific joint-data contribution. The observed-data likelihood sums over the unobserved class:

Here \(\theta\) contains the membership, longitudinal, event-risk, and any other parameters of the specified model. The diagnostic target is a maximizer of this same observed-data log-likelihood under the same constraints and using the same data and likelihood implementation. If fits use different observations, model specifications, or likelihood calculations, their log-likelihoods are not a direct comparison of optimization solutions.

The likelihood comparison assumes the optimization runs evaluate the same objective function and use compatible parameter constraints. It also assumes the reported values are finite and that convergence diagnostics are available. If the likelihood is numerically unstable, or a run terminates before approaching a stationary point, its likelihood alone is not enough to label the result a local maximum.

Repeated Starts and a Practical Comparison

A repeated-start analysis fits the same model many times, changing initial values while keeping the data, likelihood, and constraints fixed. Starts should be varied enough to explore different plausible regions of the parameter space rather than merely adding tiny perturbations to one initial value. Where appropriate, initial values should respect the model's constraints—for example, class probabilities should be valid probabilities and variance parameters should be positive.

For each run, record at least the final observed-data log-likelihood, the optimizer's convergence status, a measure of the final gradient or other stopping criterion, the parameter estimates, and any warnings. The log-likelihood should be computed at the final reported estimates. Compare the largest likelihood with the other converged fits, and then inspect whether parameter estimates agree after allowing for class-label permutations.

For a smooth likelihood at an interior local maximum, the gradient is zero and the Hessian is negative semidefinite. In computation, the gradient will rarely be exactly zero, so its size must be judged against the optimizer's scaling and tolerance. A negative-definite Hessian is evidence of a strict local maximum in unconstrained coordinates, but near-boundary solutions require care: the relevant maximum may be constrained, and ordinary unconstrained derivative checks need not apply directly.

The first condition is a stationarity check; the second expresses local maximality. Neither a small gradient nor a convergence flag alone proves that a run found a good maximum. They are useful alongside likelihood comparisons and parameter checks.

Allowing for Label Switching

Class labels are arbitrary: permuting class-specific parameters and the corresponding membership probabilities leaves the observed-data likelihood unchanged. Therefore, comparisons must first align labels. For two classes, a simple check is to compare the estimates both in their original order and after swapping all class-specific quantities consistently. With more classes, the same idea applies to possible permutations; the matching should use a prespecified, interpretable set of parameter summaries rather than one parameter chosen after seeing the results.

After alignment, examine parameter estimates and, where helpful, fitted class probabilities or other model-implied quantities. Exact equality of log-likelihoods is expected under a pure relabeling, apart from numerical rounding. A matching likelihood without matching aligned estimates may instead reflect a flat or weakly determined direction, or two distinct solutions with similar fit; it is not enough to conclude that the runs found the same parameter solution.

Parameter and Convergence Checks

A credible candidate solution should satisfy the model's parameter constraints and should not carry unresolved numerical warnings. Check that class probabilities are in the permitted range and sum to one for each relevant covariate pattern; that variance and other constrained parameters are admissible; and that estimates are finite. Check the optimizer's termination reason, gradient or score size, iteration limits, and any warnings about singular calculations. A parameter value on a boundary is not automatically invalid, but it calls for explicit scrutiny and interpretation under the fitted model's constraints.

Parameter checks can expose a run that appears to have a competitive likelihood but has not produced a usable estimate. For example, an invalid probability, nonpositive variance where positivity is required, or non-finite estimate indicates that the result must not be treated as an acceptable fit. Similarly, reaching a maximum iteration limit is not evidence of convergence merely because the likelihood changed little in the final iteration.

How to Interpret Differences and Choose Follow-Up Checks

When two valid converged runs differ, first verify that the data, model, constraints, and likelihood implementation are identical. Then align class labels and compare log-likelihoods, parameter estimates, and relevant fitted quantities. A materially lower likelihood from a valid converged fit is evidence that the optimizer found an inferior solution relative to the best run observed. Similar likelihoods but substantially different estimates warrant further investigation: the runs may be distinct solutions, or the likelihood may be relatively flat in some direction.

A useful follow-up is to start additional runs near each distinct candidate and perturb their parameter values modestly, while continuing to record convergence and likelihood. If nearby starts repeatedly return to the same candidate, that supports its numerical stability. It does not establish global optimality, since other regions may remain unexplored. If runs keep ending at different points, or if estimates change markedly while the likelihood hardly changes, report that instability rather than presenting a single fit as unambiguous.

These checks diagnose optimization; they do not settle whether the chosen number of classes is appropriate, whether classes are well separated, or whether the model fits the data adequately. Those are different questions. Likewise, information criteria compare model fit with a complexity penalty; they are not a substitute for checking whether each model's likelihood was successfully optimized.

Common Pitfalls and Limitations

• Treating a convergence flag as proof. A flag reports that software met its own stopping rule. Check the likelihood, gradient or score, parameter constraints, and warnings as well.
• Comparing class labels literally. A consistent permutation can make equivalent fits look different. Align labels before judging parameter differences.
• Declaring the largest observed likelihood global. Repeated starts explore only a finite set of initial values. The largest found is the best among those runs, not a guarantee about all admissible values.
• Calling every different estimate a local maximum. A run that stopped early or violates constraints is not an acceptable converged candidate. A distinct but valid estimate still needs local convergence checks.
• Using likelihood alone to decide sameness. Similar likelihoods can accompany different parameter values. Compare aligned estimates and fitted quantities, too.

The number and quality of starts cannot be reduced to a universal rule. A model with a complicated likelihood surface may require substantial exploration, and no finite multi-start procedure certifies that every mode has been found. The practical conclusion should therefore be calibrated: state how many starts were used, how many converged, the range of likelihoods among acceptable fits, and whether competing estimates remained.