Label Switching and Parameter Identifiability

Why Class Labels Can Be Unstable

A finite-mixture joint model represents the observed data as arising from one of several latent classes. The classes may differ in their longitudinal trajectories, event risks, or both. But the labels “class 1” and “class 2” are names assigned by the analyst: unless the model imposes an identifying convention, exchanging those names does not change the statistical model. This symmetry can make an optimizer return parameter estimates that look different while fitting the data equally well.

A separate problem occurs when the class-specific profiles are very similar. In that case, the data may provide little information about how observations should be divided among classes or about the separate parameters for each class. The likelihood can then be relatively flat in directions that change class proportions or profiles. This is weak identification, not merely a relabeling. It matters because estimates and standard errors may be unstable even after the labels have been put in a consistent order.

Mixture Likelihood and Identifiability

Let \(D_i\) denote the observed longitudinal and time-to-event data for subject \(i\), and let \(C_i\) be that subject’s unobserved class, taking values \(1,\ldots,K\). Write \(\pi_{ik}=P(C_i=k\mid X_i)\) for the probability of class \(k\), possibly conditional on baseline covariates \(X_i\), and let \(f_k(D_i\mid X_i;\theta_k)\) be the class-specific joint-data density or probability mass. The observed-data contribution for subject \(i\) is the sum over possible classes:

Here \(\Theta\) collects the class probabilities and all class-specific parameters, including trajectory and event-model parameters. The class-specific density is joint: it represents the probability model for the observed longitudinal and event data together, under the assumptions of the specified joint model. The subjects’ likelihood is the product of these contributions when subjects are independent.

Identifiability is a property of the model and its observable distribution, not a guarantee that a particular sample will yield precise estimates. A model may be identifiable in principle but weakly identified for the data at hand. Conversely, equal likelihoods obtained simply by renaming classes do not show that the model has failed to distinguish different observable distributions; they show that labels have no intrinsic ordering.

Why Permuting Labels Leaves the Likelihood Unchanged

Take any permutation \(\sigma\) of \(1,\ldots,K\). Construct a new parameter vector by assigning the old class-\(\sigma(k)\) probability and parameters to the new class \(k\): \(\pi'_{ik}=\pi_{i,\sigma(k)}\) and \(\theta'_k=\theta_{\sigma(k)}\). The resulting contribution is

The middle equality follows because \(\sigma(k)\) runs through every class index exactly once; changing the order of the terms in a finite sum does not change the sum. This holds for every subject, so the full likelihood is unchanged as well. The equality requires that the relabeling be consistent: the class probability and every parameter attached to a class—including its trajectory, event-risk parameters, and within-class random-effects specification—must move together.

This argument is the central justification for treating label permutations as equivalent representations. If the labels are exchanged in only part of the model, such as swapping trajectory parameters but not the matching event-risk parameters, the resulting model will generally be different.

Weak Identification When Class Profiles Overlap

Label symmetry persists even when classes are clearly separated. Weak identification arises when the observed data are not informative enough to separate the class-specific profiles. For example, if two classes have nearly the same longitudinal trajectories and event-risk models, their joint-data distributions will be similar. In the mixture sum, it can then be hard to determine which subjects support one class rather than the other. The class probabilities and class-specific parameters may move together with little change in the likelihood.

An exact limiting case clarifies the issue. If two classes have identical observed-data densities, \(f_1(D_i)=f_2(D_i)=f(D_i)\) for every possible \(D_i\), then their combined contribution is

The data contribution depends on the sum of those two probabilities, not on how that sum is split between the classes. Their separate class probabilities cannot be learned from these observations under this exact equality. When the densities are merely close rather than identical, the separate parameters may be estimable in principle, but only imprecisely.

Diagnosing Switching and Weak Identification

The useful first question is whether apparently different fits can be aligned by a pure permutation. Compare complete class profiles, including class probabilities and all longitudinal and event components. If one fit’s class 1 matches another fit’s class 2 across these components, and their likelihoods agree to numerical precision, label switching is a plausible explanation. If matching labels still leaves substantially different estimates, or if estimates vary widely under small perturbations, weak identification or another fitting problem may be involved.

Several signs deserve attention, interpreted together rather than as automatic proof:

• Class-specific profiles nearly coincide. Similar trajectory parameters and event-risk parameters can make the classes hard to distinguish through the joint data.
• Large uncertainty or unstable estimates. Very large standard errors, broad likelihood-based intervals, or estimates that change substantially with small data or specification changes can signal weak information. Standard errors alone are not a definitive diagnosis, especially when the likelihood is irregular.
• Highly correlated parameter estimates. For example, class probabilities and class-specific profile parameters may trade off against one another while leaving fitted observable distributions similar.
• Different parameter vectors but matching fitted distributions. Check whether differences are a label permutation first; then examine whether non-permutation differences still give nearly equivalent likelihoods and observable implications.

The observed-data likelihood, fitted class-specific profiles, and uncertainty summaries should be examined together. A single small likelihood difference or a single unstable coefficient does not identify the cause. Numerical optimization can also be imperfect, so this tutorial’s distinction is conceptual: label switching is an exact symmetry of the model, while weak identification concerns insufficient separation in the observable information. Detailed diagnosis of competing optimizer solutions is a separate issue.

Ordering Constraints and Other Responses

One common computational convention is to impose an ordering on a class-specific parameter, such as requiring the class-specific mean trajectory at a chosen time to increase with the label. This selects one representative from the set of permuted parameterizations, when the ordering is well-defined and the profiles differ on that parameter. It can make repeated output easier to compare.

An ordering convention does not create new information or resolve weak identification. If the profiles are nearly tied on the ordered parameter, small changes in the data can reverse their order. If the profiles are exactly tied there, the constraint may fail to distinguish the labels. Nor does the convention make “higher-mean class” a biologically or clinically meaningful group by itself. Labels assigned through an ordering rule should be described as a naming convention.

When reporting results, state the rule used to name classes, show the full class-specific profiles rather than only their labels, and distinguish exact label equivalence from uncertainty about the profiles themselves. If profiles overlap substantially, avoid presenting the class-specific estimates as sharply distinct patterns. A mixture model may still summarize the data usefully, but the strength of class-specific conclusions should match the information in the data.

Common Mistake and Key Takeaway

Common mistake: “The optimizer returned different class 1 estimates on two runs, so the model is not identifiable.” That conclusion is too quick. If the entire set of parameters is exchanged between classes and the likelihood is unchanged, the difference is the expected label symmetry of a mixture model. First align the labels and compare the corresponding profiles. If estimates still differ beyond permutation, examine the likelihood and uncertainty: near-overlap can create weak identification, but other numerical or model issues may also need consideration.