Deriving the Observed-Data Mixture Likelihood

Why the Observed-Data Likelihood Is a Mixture

In a latent class joint model, each subject belongs to one trajectory class, but the class label is not observed. The model specifies a probability for each possible class and, within each class, a joint contribution for the subject’s longitudinal record and event-time information. Because the actual class is unknown, the likelihood for the observed data must include every possible class membership.

The essential operation is a sum over possible classes, not a choice of the most likely class. That sum must be formed for each subject before taking logarithms or combining independent subjects. Right-censoring changes the class-specific survival contribution, but not the mixture operation.

Definition: Let \(Z_i\in\{1,\ldots,G\}\) be subject \(i\)’s unobserved class, and let \(w_i\) denote baseline covariates used to model class membership. Write \(\pi_{ig}=P(Z_i=g\mid w_i)\), where \(\pi_{ig}\geq 0\) and \(\sum_{g=1}^{G}\pi_{ig}=1\). Let \(Y_i\) be the subject’s observed longitudinal record, \(t_i=\min(T_i,C_i)\) the observed event-or-censoring time, and \(\delta_i=I(T_i\leq C_i)\) the event indicator. The class-specific joint likelihood contribution is \(J_{ig}=p(Y_i,t_i,\delta_i\mid Z_i=g)\). The observed-data contribution is \(L_i=\sum_{g=1}^{G}\pi_{ig}J_{ig}\).

Here \(T_i\) is the event time and \(C_i\) is the censoring time. The symbol \(p\) denotes a density or probability as appropriate to the observed data. Thus, a likelihood contribution involving a continuous event time is a density contribution, not the probability of observing that exact time. The parameters in both \(\pi_{ig}\) and \(J_{ig}\) are understood to be evaluated at their current values.

From a Hidden Class to the Mixture Formula

If the class were observed, subject \(i\)’s contribution would be \(\pi_{ig}J_{ig}\) when \(Z_i=g\): the class-membership probability multiplied by the data contribution in that class. With \(Z_i\) unobserved, the possible class states are mutually exclusive and collectively exhaustive. The observed-data likelihood therefore adds their contributions.

L_i =\sum_{g=1}^{G}p(Z_i=g\mid w_i)\,p(Y_i,t_i,\delta_i\mid Z_i=g) =\sum_{g=1}^{G}\pi_{ig}J_{ig}.

This identity follows by partitioning the observed-data density according to the possible values of \(Z_i\). In particular, it does not require selecting a class first. It also does not require that the class-specific contribution factor into separate longitudinal and survival terms: \(J_{ig}\) can be the full joint contribution specified by the model.

\begin{aligned} p(Y_i,t_i,\delta_i\mid w_i) &=\sum_{g=1}^{G}p(Y_i,t_i,\delta_i,Z_i=g\mid w_i)\\ &=\sum_{g=1}^{G}p(Z_i=g\mid w_i)\, p(Y_i,t_i,\delta_i\mid Z_i=g,w_i). \end{aligned}

The second equality uses the probability product rule. If the model includes \(w_i\) in the class-specific data distribution as well as in class membership, that dependence is included in \(J_{ig}\). When subjects are independent conditional on the model parameters and their covariates, the cohort likelihood is the product of their observed-data contributions. Its log-likelihood is a sum of logarithms of those subject-level mixtures.

L=\prod_{i=1}^{n}\left(\sum_{g=1}^{G}\pi_{ig}J_{ig}\right), \qquad \ell=\sum_{i=1}^{n}\log\left(\sum_{g=1}^{G}\pi_{ig}J_{ig}\right).

The order matters: first sum across classes for each subject, then take the logarithm, then sum across subjects. In general, the logarithm of a sum is not the sum of logarithms.

Including an Event or Right-Censoring

For a continuous event time, let \(S_{ig}(t)\) be the class-specific survival function and \(\lambda_{ig}(t)\) the corresponding hazard. The event-time density is \(f_{ig}(t)=\lambda_{ig}(t)S_{ig}(t)\). If the model specifies conditional independence of the longitudinal record and event process within class, the class-specific joint contribution has the familiar form below. This factorization is an assumption of that specification; when the model instead specifies within-class dependence, use its full joint contribution \(J_{ig}\).

Model-specific form: Under within-class conditional independence, with an ignorable censoring mechanism, \(J_{ig}=f_g(Y_i)\{\lambda_{ig}(t_i)\}^{\delta_i}\{S_{ig}(t_i)\}^{1-\delta_i}\). Here \(f_g(Y_i)\) is the class-specific longitudinal likelihood contribution. The censoring mechanism is omitted when it does not contain parameters of interest and is independent of the event and longitudinal data under the model’s conditioning assumptions.

When \(\delta_i=1\), the subject experienced the event at \(t_i\), so the survival contribution is its event-time density, \(\lambda_{ig}(t_i)S_{ig}(t_i)\). When \(\delta_i=0\), the subject was known to remain event-free through \(t_i\), so the contribution is the survival probability \(S_{ig}(t_i)\). The exponent notation combines the cases without changing their meanings.

J_{ig} =f_g(Y_i) \begin{cases} \lambda_{ig}(t_i)S_{ig}(t_i), & \delta_i=1,\\ S_{ig}(t_i), & \delta_i=0 \end{cases} =f_g(Y_i)\{\lambda_{ig}(t_i)\}^{\delta_i} \{S_{ig}(t_i)\}^{1-\delta_i}.

This expression assumes a continuous event-time distribution; if event times have point masses, the event contribution must use the appropriate probability mass instead of a density. It also assumes that censoring can be treated as noninformative for the likelihood being fitted, conditional on the variables and model structure specified. If censoring carries additional information about the event process, simply omitting its mechanism is not justified.

Worked Example: An Event Observed in One of Two Classes

Suppose a hypothetical subject has class probabilities \(\pi_{i1}=0.70\) and \(\pi_{i2}=0.30\). The subject has a longitudinal likelihood contribution of \(0.12\) in class 1 and \(0.05\) in class 2. Assume the model’s within-class factorization and an exponential event model, with hazards \(0.08\) and \(0.15\) per month, respectively. The event is observed at \(t_i=4\) months.

For a constant hazard \(\lambda\), \(S(t)=\exp(-\lambda t)\). Thus, for class 1, \(S_{i1}(4)=\exp(-0.08\times4)=\exp(-0.32)=0.7261\). Its event contribution is \(J_{i1}=0.12\times0.08\times0.7261=0.006971\).

For class 2, \(S_{i2}(4)=\exp(-0.15\times4)=\exp(-0.60)=0.5488\). Its event contribution is \(J_{i2}=0.05\times0.15\times0.5488=0.004116\).

Weight these contributions by the class probabilities and add: \(L_i=0.70(0.006971)+0.30(0.004116)=0.004880+0.001235=0.006115\), up to rounding. This is the subject’s observed-data likelihood contribution under the stated model. The event density includes both the hazard at month 4 and survival to month 4; using only the hazard would omit the probability of remaining event-free until then.

Worked Example: A Right-Censored Subject

Consider a different hypothetical subject with \(\pi_{i1}=0.60\) and \(\pi_{i2}=0.40\), whose event time is right-censored at 5 months. Suppose the longitudinal contributions are \(0.10\) and \(0.08\), and the class-specific constant hazards are \(0.04\) and \(0.12\) per month. The censoring is assumed ignorable, and the within-class factorization is assumed to hold.

Because the subject is censored, \(\delta_i=0\), and each class contributes the longitudinal likelihood times survival through month 5. In class 1, \(S_{i1}(5)=\exp(-0.04\times5)=\exp(-0.20)=0.8187\), so \(J_{i1}=0.10\times0.8187=0.08187\).

In class 2, \(S_{i2}(5)=\exp(-0.12\times5)=\exp(-0.60)=0.5488\), so \(J_{i2}=0.08\times0.5488=0.04390\). Therefore, \(L_i=0.60(0.08187)+0.40(0.04390)=0.04912+0.01756=0.06668\), up to rounding.

No hazard at month 5 is multiplied into this contribution: the observed information is that the subject survived through month 5, not that an event occurred at that time. Replacing \(S_{ig}(5)\) with \(\lambda_{ig}(5)S_{ig}(5)\) would treat a censored subject as if the event had been observed.

Why the Sum Must Be Taken Before the Logarithm

The mixture is a sum of likelihood contributions, not a weighted average of log-likelihoods. This distinction is consequential because the class is a single unobserved state for a subject: the observed record could have arisen under any of the classes, with the corresponding probability and class-specific likelihood.

Worked Example: Combining Contributions Across Two Subjects

Suppose two hypothetical subjects have the following class probabilities and class-specific joint contributions, calculated from their respective longitudinal and event records under a fitted model. For subject 1, \((\pi_{11},\pi_{12})=(0.60,0.40)\) and \((J_{11},J_{12})=(0.08,0.02)\). For subject 2, \((\pi_{21},\pi_{22})=(0.25,0.75)\) and \((J_{21},J_{22})=(0.03,0.06)\).

For subject 1, \(L_1=0.60(0.08)+0.40(0.02)=0.048+0.008=0.056\). For subject 2, \(L_2=0.25(0.03)+0.75(0.06)=0.0075+0.045=0.0525\).

If the subjects are independent conditional on the model parameters, the cohort likelihood is \(L=L_1L_2=0.056\times0.0525=0.002940\), and the log-likelihood is \(\ell=\log(0.002940)\approx-5.829\). The product is across subjects; the sum inside each \(L_i\) is across that subject’s possible classes.

In contrast, assigning each subject to a class first would discard the uncertainty represented by the other class’s contribution. Likewise, putting a single class sum around the product of both subjects’ contributions would imply a shared class for the pair, which is a different model from subject-specific class membership.

Assumptions, Diagnostics, and Alternatives

The mixture identity itself is a probability decomposition. The assumptions enter through the model used to calculate \(\pi_{ig}\) and \(J_{ig}\), and through the construction of the cohort likelihood. Check that the class probabilities are nonnegative and sum to one for every subject; that every class-specific contribution uses the correct event or censoring information; and that the independence assumptions used to multiply subject contributions are appropriate. For the factorized contribution, also check the stated within-class independence assumption and the assumptions that make censoring ignorable.

A useful implementation check is to calculate a few \(J_{ig}\) values and independently reconstruct \(L_i=\sum_g\pi_{ig}J_{ig}\). Confirm that an event uses the event density, a censored record uses survival, and the class weights are applied once. Numerical calculations can underflow when many small densities are multiplied; evaluating the cohort log-likelihood as a sum of subject-level log-mixtures is more stable, with log-sum-exp computation used when necessary.

The main alternative is hard class assignment: choose one class for each subject and use only that class’s contribution. This is not the observed-data likelihood because it treats an unobserved label as known and discards contributions from other possible labels. A separate modeling alternative is to represent unobserved heterogeneity using continuous random effects rather than a finite set of classes; that changes the latent structure and the likelihood calculation. Neither alternative changes the basic principle that unobserved states must be marginalized according to the model when forming an observed-data likelihood.

Common pitfall: It is incorrect to say, “This subject is most likely in class 1, so I will use only \(J_{i1}\).” A most likely class is not an observed class. The correction is to retain all class contributions and calculate \(L_i=\sum_g\pi_{ig}J_{ig}\); selecting a class first changes the likelihood and loses uncertainty about membership.

The mixture likelihood also has limits. Its validity depends on the chosen number and interpretation of classes, the class-membership model, and the class-specific longitudinal and survival distributions. A correctly computed likelihood does not by itself establish that those modeling choices describe the data well. Poor class-specific fit, an unsuitable censoring assumption, or dependence between subjects not represented in the model can invalidate the analysis even when the summation is implemented correctly.

Check Your Understanding

Work through the questions, then compare your results with the solutions.

For two classes, \(\pi_{i1}=0.35\), \(\pi_{i2}=0.65\), \(J_{i1}=0.04\), and \(J_{i2}=0.02\). What is \(L_i\)?
A subject is censored at time \(c\). Under the factorized model, which survival contribution belongs in \(J_{ig}\), and why is the hazard not included?
For two independent subjects, should their class-specific contributions be summed over classes together before multiplying, or should each subject’s class contributions be summed first? Explain.
State one assumption needed to omit the censoring mechanism from the likelihood.

Solutions. (1) \(L_i=0.35(0.04)+0.65(0.02)=0.014+0.013=0.027\). (2) Use \(S_{ig}(c)\), because the observed record establishes survival through \(c\) but does not establish an event at \(c\); a hazard factor is used for an observed event. (3) Sum within each subject first, then multiply: \(L=L_1L_2\). A single sum around both subjects’ contributions would generally represent a shared class rather than separate subject-specific classes. (4) For example, censoring must be independent of the event process conditional on the variables and model structure used, so that the censoring mechanism contributes no parameter-dependent information to the likelihood of interest.