Label Switching and Class Ordering

Why Class Labels Need a Convention

A two-class mixture model can describe one class as having faster clearance and the other as having slower clearance. The labels “fast” and “slow” are useful descriptions, but the likelihood does not inherently know which class is called which. If every class-specific quantity and membership probability is exchanged consistently, the model describes exactly the same distribution of observations.

This symmetry matters when fitting or reporting a model. Two fits may have matching likelihoods and predictions while listing different parameter estimates under “class 1” and “class 2.” That difference alone is not evidence that the fits disagree. A reproducible ordering rule, such as defining the fast class to be the class with higher typical clearance, gives the labels a consistent interpretation.

The Two-Class Likelihood and Its Symmetry

Let \(y_i\) denote all observations for subject \(i\). Let \(F\) and \(S\) denote the fast- and slow-clearance classes, and let \(\pi_{iF}\) and \(\pi_{iS}\) be that subject’s class-membership probabilities, with \(\pi_{iF}+\pi_{iS}=1\). Let \(L_{iF}\) and \(L_{iS}\) be the likelihood contributions for the subject’s observations under each class. In an NLME model, each class-specific contribution accounts for the within-class model, including the random-effects distribution as specified by the model.

The mixture likelihood uses this sum because the class is latent: the observed data do not come with a known class label. The contributions for independent subjects are combined across subjects, commonly by multiplying their marginal likelihood contributions, or equivalently by summing their log contributions.

The label symmetry follows directly. Exchange the class-specific likelihoods and exchange their weights:

Thus, if the entire parameterization is swapped, every subject’s likelihood contribution remains unchanged. The total likelihood remains unchanged as well. This is an exact property of the mixture representation, not an approximation or a consequence of a particular data set.

The model must be relabeled coherently. Swapping only the clearance values, for example, while leaving the membership probabilities attached to their old classes would generally change the likelihood. A valid swap exchanges each class’s full set of parameters and its membership probability, along with any class-specific covariate effects.

How to Order the Classes

For clearance classes, a natural convention is to define \(F\) as the class with higher typical clearance and \(S\) as the class with lower typical clearance. The word “typical” refers to the population parameter specified by the model, not an individual subject’s estimated clearance. The exact parameter being ordered should be stated, including the covariate values at which it is defined if the model includes clearance covariates.

This ordering selects one representative from two equivalent labelings. It does not add information to the observations, change the class-specific predictions, or improve separation of the classes. If the two clearances are equal, the ordering cannot distinguish the labels; at equality, the two class labels are interchangeable under this rule.

There are two common implementations. One may fit the unconstrained, symmetric model and then relabel the fitted classes for reporting. Alternatively, an implementation may impose an ordering during estimation or use a parameterization that guarantees it. If the ordering is only a naming convention applied after fitting, it does not alter the fitted likelihood. If an optimization constraint is imposed, it should represent the same parameter space up to relabeling; constraints that restrict additional parameter relationships can change the model.

Relabeling Membership Models

When class probabilities depend on covariates, the membership model must be transformed along with the class labels. For example, suppose the fast-class probability follows a logistic model with covariate vector \(z_i\):

After exchanging the labels, the probability called the new fast-class probability is the old slow-class probability. Since \(1-\pi_{iF}\) is the logistic function evaluated at the negative of its linear predictor, the intercept and covariate coefficients change sign in this binary logistic representation. The class-specific likelihood parameters are exchanged at the same time.

Why the Ordering Preserves the Fit

The invariance can be stated as a simple reparameterization argument. Let \(\theta\) denote all model parameters, including class-specific parameters and membership-model parameters. Define \(T(\theta)\) to be the parameter vector obtained by exchanging the class labels everywhere. Applying the exchange twice returns the original parameter vector, so \(T(T(\theta))=\theta\). For every subject, the sum defining the marginal contribution is unchanged under this transformation. Therefore the full likelihood satisfies \(\mathcal{L}(T(\theta))=\mathcal{L}(\theta)\).

If a fitted parameter vector has \(CL_F<CL_S\), applying \(T\) gives an equivalent vector with the order reversed. Consequently, selecting the representative with \(CL_F>CL_S\) removes the duplicate labeling, while preserving the likelihood value and fitted distribution. This is the mathematical reason that a post-fit ordering convention does not change the fitted model.

The argument assumes a genuine exchangeability of labels: both classes have the same structural role in the model, apart from their class-specific parameters and probabilities. If the model has a feature that distinguishes labels by definition, or if a parameter constraint treats the labels differently, the simple symmetry may not hold. Before using the ordering argument, verify that every class-specific term and probability is exchanged consistently and that no asymmetric constraint has been added.

Ordering With Covariates, Diagnostics, and Alternatives

If clearance depends on covariates, specify where the ordering is evaluated. An ordering at a reference covariate value guarantees \(CL_F>CL_S\) there, but class-specific covariate effects can make the clearances cross elsewhere. If the intended interpretation is “higher clearance throughout a covariate range,” the fitted clearance functions must be compared across that range; a single reference-point ordering does not establish that stronger statement.

Useful checks focus on whether the relabeling is coherent and reproducible:

• Compare fitted likelihoods and predictions before and after relabeling. They should agree to numerical precision if only labels have changed.
• Check that membership probabilities sum to one and that the probability assigned to each relabeled class matches its original counterpart.
• Keep a record of the ordering parameter and reference covariate values used for reporting.
• When comparing results from different fits, order each fit by the same rule before comparing class-specific estimates.
• If a class ordering flips across fits, inspect whether it is merely a label swap or whether the estimated class values and predictions also differ.

A common alternative is to retain arbitrary labels such as class 1 and class 2 and report the estimates without ordering. That is mathematically valid, but comparisons across fits can become confusing because the labels have no stable interpretation. Post-fit relabeling is usually the simplest reporting convention. Enforcing an order during optimization can be convenient, but it requires care: an implementation must not accidentally exclude parameter values beyond the label duplication or introduce an unintended boundary restriction. Neither approach solves a lack of information about the class-specific parameters; ordering only fixes names.