A Worked Two-Class Likelihood Calculation

Why calculate a subject likelihood?

In a latent class joint model, a subject’s class is not directly observed. Each possible class assigns a probability to the observed longitudinal measurements and event-time information. The subject’s likelihood contribution combines those class-specific probabilities, weighted by the class membership probabilities. Bayes’ rule then turns the same quantities into posterior class probabilities.

This tutorial works through that calculation for two classes. The arithmetic is simple enough to check by hand, but it is important to keep the pieces straight: a repeated-measurement contribution, an event-time contribution, the class probability, and—when needed—the censoring contribution. All numerical data below are hypothetical.

Model and likelihood components

Definition: Let \(C_i\in\{1,2\}\) denote subject \(i\)’s latent class, with class probabilities \(\pi_{ik}=P(C_i=k\mid x_i)\), where \(x_i\) denotes modeled covariates and \(\pi_{i1}+\pi_{i2}=1\). Let \(y_{ij}\) be the \(j\)th longitudinal measurement for subject \(i\), observed at time \(t_{ij}\). Write \(T_i\) for the event time, \(C_i^{\mathrm{cens}}\) for the censoring time, \(u_i=\min(T_i,C_i^{\mathrm{cens}})\), and \(\delta_i=I(T_i\leq C_i^{\mathrm{cens}})\). The observed event-time data are \((u_i,\delta_i)\); \(\delta_i=1\) means the event was observed, and \(\delta_i=0\) means the event time was right-censored.

For a basic calculation, suppose that, conditional on class \(k\) and the modeled covariates, the measurements are independent Gaussian variables with means \(m_{ijk}\) and standard deviation \(\sigma_k\). Then

L^{(Y)}_{ik} = \prod_{j=1}^{n_i} \frac{1}{\sqrt{2\pi}\sigma_k} \exp\left\{-\frac{(y_{ij}-m_{ijk})^2}{2\sigma_k^2}\right\}.

This product assumption is a simplifying model specification, not a consequence of having repeated measurements. If measurements are conditionally dependent through random effects, the class-specific longitudinal likelihood must account for that dependence; for example, it may be obtained by integrating the product of conditional densities over the random-effects distribution. Multiplying marginal densities as though the measurements were independent would then be incorrect.

Let \(h_k(u_i)\) be the event hazard in class \(k\) at time \(u_i\), and let \(H_k(u_i)\) be its cumulative hazard up to that time. The event-time contribution is

L^{(T)}_{ik} = [h_k(u_i)]^{\delta_i}\exp\{-H_k(u_i)\}.

For an observed event, this is the event density \(h_k(u_i)\exp\{-H_k(u_i)\}\). For a right-censored observation, it is the survival probability \(\exp\{-H_k(u_i)\}\). In particular, a censored observation does not contribute an event density at the censoring time.

The censoring mechanism needs separate attention when calculating posterior class probabilities. Suppose \(T_i\) and \(C_i^{\mathrm{cens}}\) are independent conditional on class and modeled covariates. Let \(f_{C,k}\) and \(S_{C,k}\) denote the censoring-time density and survival function in class \(k\). The censoring part of the observed-data likelihood is

L^{(C)}_{ik} = [S_{C,k}(u_i)]^{\delta_i} [f_{C,k}(u_i)]^{1-\delta_i}.

For an observed event at \(u_i\), the subject must remain uncensored through that time, giving \(S_{C,k}(u_i)\). For a censored observation at \(u_i\), the censoring time has density \(f_{C,k}(u_i)\). If the censoring distribution is the same across classes conditional on the modeled covariates, this factor is common to both class contributions and cancels from the posterior class probabilities. If it differs by class, it generally does not cancel and must be included for that calculation. A censoring factor that contains no event-model parameters may be omitted when estimating event-model parameters under the relevant independent-censoring assumptions; that does not by itself make the factor ignorable for posterior class probabilities.

Mixture likelihood and posterior class probability

Assume the longitudinal and event-time data are conditionally independent given class and modeled covariates, with any additional shared latent variables included or integrated out as specified by the model. Then a class-specific subject contribution is

L_{ik} = L^{(Y)}_{ik}L^{(T)}_{ik}L^{(C)}_{ik}.

If the censoring factor is common across classes, it can be omitted from this expression when calculating posterior class probabilities, because it multiplies every class contribution by the same amount. The observed-data likelihood contribution for the subject is the mixture

L_i=\sum_{k=1}^{2}\pi_{ik}L_{ik}.

The posterior probability of class \(k\), conditional on the observed data and modeled covariates, is

P(C_i=k\mid \text{observed data},x_i) = \frac{\pi_{ik}L_{ik}}{\sum_{\ell=1}^{2}\pi_{i\ell}L_{i\ell}}.

This follows directly from Bayes’ rule: the numerator is the joint probability contribution for class \(k\) and the observed data, and the denominator sums those contributions over the two mutually exclusive classes. The posterior probabilities are nonnegative and sum to one.

Worked Example: Repeated Measurements and a Censored Event Time

Consider one hypothetical subject with two measurements, \(y_{i1}=1.2\) and \(y_{i2}=1.8\). Both classes have Gaussian standard deviation \(\sigma_k=0.5\). The class-specific means are \(m_{i11}=1.0,\ m_{i21}=2.0\) in class 1 and \(m_{i12}=0.5,\ m_{i22}=1.5\) in class 2. The class probabilities are \(\pi_{i1}=0.6\) and \(\pi_{i2}=0.4\). The event time is censored at \(u_i=8\). Assume exponential event hazards \(\lambda_1=0.10\) and \(\lambda_2=0.04\) per year, and assume the censoring distribution is common across classes, so its factor cancels in the posterior calculation.

Longitudinal contribution for class 1. The residuals are \(1.2-1.0=0.2\) and \(1.8-2.0=-0.2\). Each Gaussian density has normalizing factor \(1/(\sqrt{2\pi}\,0.5)=0.7978846\). Therefore,

L^{(Y)}_{i1} = (0.7978846e^{-0.2^2/(2(0.5)^2)}) (0.7978846e^{-(-0.2)^2/(2(0.5)^2)}) \approx 0.5424916.

Longitudinal contribution for class 2. The residuals are \(1.2-0.5=0.7\) and \(1.8-1.5=0.3\), so

L^{(Y)}_{i2} = (0.7978846e^{-0.7^2/(2(0.5)^2)}) (0.7978846e^{-0.3^2/(2(0.5)^2)}) \approx 0.1995715.

Event-time contributions. For an exponential hazard \(\lambda_k\), \(H_k(u)=\lambda_k u\). Since the event is censored, \(\delta_i=0\), so the contribution is survival through year 8:

L^{(T)}_{i1}=e^{-0.10(8)}=e^{-0.8}\approx0.4493290, \qquad L^{(T)}_{i2}=e^{-0.04(8)}=e^{-0.32}\approx0.7261490.

Combine the factors and calculate the posterior. Omitting the common censoring factor, the weighted contributions are

\begin{aligned} \pi_{i1}L^{(Y)}_{i1}L^{(T)}_{i1} &=0.6(0.5424916)(0.4493290)\approx0.1462543,\\ \pi_{i2}L^{(Y)}_{i2}L^{(T)}_{i2} &=0.4(0.1995715)(0.7261490)\approx0.0579675. \end{aligned}

Thus the mixture likelihood contribution, up to the common censoring factor, is \(0.1462543+0.0579675=0.2042218\). The posterior probability of class 1 is \(0.1462543/0.2042218\approx0.7162\), and that of class 2 is \(0.0579675/0.2042218\approx0.2838\). Given this model and these observations, class 1 has the larger posterior probability; the result is not a certainty about the subject’s unobserved class.

Why the censoring factor can matter

The cancellation condition is substantive: the censoring factor can be dropped from a class posterior only if it is common across the classes being compared, or if an appropriate conditioning scheme makes it common. Conditional independence of event and censoring times within each class does not guarantee that their censoring distributions are equal across classes.

Worked Example: A Class-Specific Censoring Distribution

Suppose a hypothetical subject is censored at year 5, with equal class probabilities. The event hazards are exponential, with \(\lambda_1=0.20\) and \(\lambda_2=0.10\) per year. The censoring times are also exponential, but their rates are \(\mu_1=0.05\) and \(\mu_2=0.30\) per year. Assume event and censoring times are independent conditional on class.

The event-time survival contributions are \(S_{T,1}(5)=e^{-1}\approx0.3678794\) and \(S_{T,2}(5)=e^{-0.5}\approx0.6065307\). Because this subject is censored, the censoring contributions are densities: \(f_{C,k}(5)=\mu_ke^{-5\mu_k}\). Hence,

f_{C,1}(5)=0.05e^{-0.25}\approx0.0389400, \qquad f_{C,2}(5)=0.30e^{-1.5}\approx0.0669390.

With equal class probabilities, the unnormalized contributions are \(0.5(0.3678794)(0.0389400)\approx0.007163\) and \(0.5(0.6065307)(0.0669390)\approx0.0203003\). The class 2 posterior is therefore \(0.0203003/(0.007163+0.0203003)\approx0.7392\). If the censoring factors were incorrectly omitted, the class 2 posterior would instead be \(0.6065307/(0.3678794+0.6065307)\approx0.6225\). Since the censoring distributions differ by class, that omission changes the posterior calculation.

Log-scale calculation and checks

Worked Example: Checking the Mixture on the Log Scale

Return to the first hypothetical subject. The two weighted class contributions were approximately \(a_1=0.1462543\) and \(a_2=0.0579675\), giving \(L_i=a_1+a_2=0.2042218\). The log mixture likelihood is

\log L_i=\log(0.2042218)\approx-1.5885.

The same result can be checked by the log-sum-exp identity:

\log(a_1+a_2) = \log(a_1)+\log\{1+\exp[\log(a_2)-\log(a_1)]\}.

Here \(\log(a_1)\approx-1.9224\), \(\log(a_2)\approx-2.8479\), and their difference is approximately \(-0.9255\). Therefore, \(-1.9224+\log(1+e^{-0.9255})\approx-1.5885\). This agrees with taking the logarithm after summing the original-scale contributions. In larger models, log-scale calculations reduce numerical underflow, but the class contributions must still be added on the probability scale or combined with a valid log-sum-exp calculation; their log likelihoods are not added directly.

Assumptions, diagnostics, and alternatives

Before interpreting a calculation, check that each factor matches the data and model. Verify that the class probabilities are nonnegative and sum to one; that each Gaussian density uses the stated mean and standard deviation; and that the event contribution matches \(\delta_i\). For \(\delta_i=0\), use survival, not density. Confirm that the event and longitudinal factors are conditionally independent as specified, or that dependence through random effects or other shared latent quantities has been handled. Finally, establish whether the censoring factor is common across classes before dropping it from a class posterior.

Several checks can reveal errors. Class-specific likelihood contributions and posterior probabilities must be nonnegative, and the posterior probabilities must sum to one. A class with a smaller weighted contribution must not receive a larger posterior probability in a two-class calculation. Recompute products using unrounded intermediate values if the final posterior is sensitive to rounding. If probabilities underflow in a larger calculation, use log likelihoods and log-sum-exp.

The mixture likelihood is the standard marginal likelihood when class membership is unobserved: it averages over possible classes using their probabilities. It is not the same as selecting the class with the largest likelihood and treating that assignment as known. Nor is a posterior class probability the same as a class membership probability before observing the subject’s data. The former uses both the prior class probability and the subject-specific likelihood.

The calculation also has limits. A posterior probability is conditional on the specified model and its estimated or supplied parameters. A numerical answer cannot verify that two classes are adequate, that the Gaussian trajectories are credible, or that the censoring assumptions hold. Diagnostics and sensitivity analyses are needed to assess those modeling choices; the arithmetic itself does not establish them.

Common mistake

A frequent error is to say, “Censoring is independent within each class, so the censoring likelihood can always be ignored when calculating class posteriors.” The first clause does not imply that the censoring distribution is identical across classes. For an observed censored time, a class-specific censoring density can favor one class over another, as the worked calculation shows. The correction is to include the censoring factor when it differs by class, or to state and justify the assumption under which it is common and cancels. A separate, parameter-estimation argument for omitting a nuisance censoring factor does not automatically justify omitting it from posterior class probabilities.

Key takeaway: For each class, multiply its class probability by the longitudinal contribution, the event-time contribution, and any censoring contribution that is not common across classes. Sum these weighted contributions for the mixture likelihood, then divide each by the sum to obtain posterior class probabilities.

Check Your Understanding

Work through each question, then compare with the solutions below.

A subject has two observed measurements and an event time censored at \(u=4\). Under an exponential event hazard \(\lambda=0.15\) per year, what is the event-time likelihood contribution?
Two classes have weighted likelihood contributions \(0.12\) and \(0.08\). What is the posterior probability of class 1?
Why can a class-specific censoring density affect a posterior class probability for a censored subject?
For an observed event, \(\delta=1\), what censoring factor appears in the observed-data likelihood under independent event and censoring times?
What modeling issue arises if repeated measurements are dependent through a subject-specific random effect but their marginal Gaussian densities are multiplied as if independent?

Solutions. (1) Censoring contributes survival: \(e^{-0.15(4)}=e^{-0.6}\approx0.5488\). (2) The denominator is \(0.12+0.08=0.20\), so the posterior for class 1 is \(0.12/0.20=0.60\). (3) Its density multiplies the class-specific event and longitudinal contributions; if that factor differs between classes, it does not cancel in the posterior ratio. (4) The censoring factor is \(S_C(u)\), the probability of remaining uncensored through the observed event time. (5) The product incorrectly imposes conditional independence; the joint longitudinal likelihood must represent the dependence, such as by conditioning on and then integrating over the random effect.