Deriving the Observed-Data Likelihood by Marginalization

Why Marginalization Is Needed

A likelihood for incomplete longitudinal data must describe what was actually recorded, not an imagined complete record. If a response pattern reveals only part of a participant’s outcome history, the unobserved outcomes are not additional observed data. Their possible values must be summed over or integrated out of the full-data model. The response pattern itself is observed and remains part of the likelihood contribution.

The selection factorization provides a direct starting point. It writes the joint law of an outcome history and its response pattern as an outcome distribution multiplied by a response distribution conditional on the full outcome history. Marginalization then removes the unobserved outcomes while preserving both the observed outcomes and the observed response indicators. No assumption that missingness is ignorable is required for this basic calculation.

Definition: Let $Y=(Y_1,\ldots,Y_T)$ be a complete outcome history and $R=(R_1,\ldots,R_T)$ its response-pattern vector, where $R_t=1$ means $Y_t$ is observed and $R_t=0$ means it is missing. For a realized pattern $r$, write $y_{\mathrm{obs}}(r)$ for the observed components and $y_{\mathrm{mis}}(r)$ for the unobserved components. Under the selection factorization, $$ f_{\theta,\psi}(y,r)=f_\theta(y)g_\psi(r\mid y). $$ Here $f_\theta(y)$ is the full-data outcome density or mass function, and $g_\psi(r\mid y)$ is the response-pattern probability conditional on the complete outcome history. The observed-data likelihood contribution is $$ L_{\mathrm{obs}}(\theta,\psi;y_{\mathrm{obs}},r) = \int f_\theta\!\left(y_{\mathrm{obs}},y_{\mathrm{mis}}\right) g_\psi\!\left(r\mid y_{\mathrm{obs}},y_{\mathrm{mis}}\right) \,d\nu(y_{\mathrm{mis}}), $$ where the integral is over the support of the missing components. For discrete missing outcomes, this integral means a sum.

The contribution is a joint density or probability for the observed outcomes and their response pattern. In particular, it is not generally just the outcome density $f_\theta(y_{\mathrm{obs}})$, nor is it generally the outcome density conditional on $R=r$. If the sampling design conditions on a response pattern, a conditional likelihood may be appropriate, but it is a different likelihood and must be derived for that design.

Deriving the Observed-Data Contribution

Fix a response pattern $r$. Every complete history compatible with the observed record has the same observed components $y_{\mathrm{obs}}$, but can have different values of $y_{\mathrm{mis}}$. The probability or density of the observed record is therefore the sum or integral of the full-data joint law over all such compatible histories. Substituting the selection factorization gives the formula in the definition.

This is ordinary marginalization of a joint distribution. Its key feature here is that the response probability may depend on the missing outcomes. Consequently, $g_\psi(r\mid y_{\mathrm{obs}},y_{\mathrm{mis}})$ generally stays inside the sum or integral. Removing it from the marginalization requires a reason, such as a response probability that does not vary with the missing outcomes over the relevant support.

Identify the recorded pattern.
Use $r$ to determine which components of $y$ are observed and which are missing.

Write the full-data joint law.
Use $f_\theta(y)g_\psi(r\mid y)$, keeping the response probability conditional on the complete history.

Marginalize only the missing outcomes.
Sum over discrete missing values or integrate over continuous missing values, holding $y_{\mathrm{obs}}$ and $r$ fixed.

Check the resulting contribution.
Confirm that the response pattern has not been dropped and that the integration or summation covers the full support of the missing outcomes.

Worked Examples

Worked Example: Summing Over a Missing Binary Outcome

Suppose two binary outcomes have joint probabilities $P(Y_1=0,Y_2=0)=0.30$, $P(Y_1=0,Y_2=1)=0.20$, $P(Y_1=1,Y_2=0)=0.10$, and $P(Y_1=1,Y_2=1)=0.40$. These probabilities sum to 1. For response pattern $r=(1,0)$, $Y_1$ is observed and $Y_2$ is missing. Suppose the response probabilities for this pattern are $0.80, 0.60, 0.50,$ and $0.30$, respectively, for the four outcome pairs listed above. Find the likelihood contribution when $Y_1=0$ is observed with this pattern.

The compatible complete histories are $(0,0)$ and $(0,1)$. For each, multiply its outcome probability by the probability of the observed response pattern, then add:

L_{\mathrm{obs}} = P(Y_1=0,Y_2=0)P(R=(1,0)\mid Y=(0,0)) + P(Y_1=0,Y_2=1)P(R=(1,0)\mid Y=(0,1)).

Substituting the values gives

L_{\mathrm{obs}}=(0.30)(0.80)+(0.20)(0.60)=0.24+0.12=0.36.

Thus the joint probability of observing $Y_1=0$ together with pattern $(1,0)$ is $0.36$. The response probability differs across the two possible missing outcomes, so it cannot be replaced by one common factor before summing.

Worked Example: Integrating a Continuous Outcome Under Selection

Let a single outcome satisfy $Y\sim N(0,1)$, and let $R=1$ denote that it is observed. Suppose $P(R=1\mid Y=y)=\Phi(0.5+y)$, where $\Phi$ is the standard normal cumulative distribution function. Find the likelihood contribution when $R=0$, so that no value of $Y$ is recorded.

The observed record contains only $R=0$, so the contribution is the integral over all possible values of $Y$:

L_{\mathrm{obs}} = \int_{-\infty}^{\infty} \phi(y)\{1-\Phi(0.5+y)\}\,dy.

To evaluate it, introduce an independent $\varepsilon\sim N(0,1)$ and represent $R=1$ as the event $\varepsilon\leq 0.5+Y$. Then $R=0$ is the event $\varepsilon-Y>0.5$. Since $\varepsilon-Y\sim N(0,2)$,

P(R=0)=1-\Phi\!\left(\frac{0.5}{\sqrt{2}}\right) =1-\Phi(0.3536) \approx 1-0.6382 =0.3618.

The observed-data contribution is therefore $0.3618$. This calculation integrates over the unobserved $Y$; it does not assign $Y$ a value or discard the response indicator. The selection probability depends on $Y$, so replacing the integral by an outcome-independent response probability would change the model.

Worked Example: Simplification When the Response Probability Uses Observed History

Suppose $Y_1,Y_2$ are binary with joint probabilities $P(0,0)=0.30$, $P(0,1)=0.10$, $P(1,0)=0.20$, and $P(1,1)=0.40$. For pattern $r=(1,0)$, assume $P(R=r\mid Y_1,Y_2)=q(Y_1)$, where $q(0)=0.25$ and $q(1)=0.50$. Find the contribution for the observed record $Y_1=1,R=r$.

The missing value $Y_2$ can be 0 or 1. Direct marginalization gives

L_{\mathrm{obs}} =(0.20)(0.50)+(0.40)(0.50) =0.10+0.20 =0.30.

Here the response probability is the same for both possible values of the missing $Y_2$, because it depends only on the observed $Y_1$. It can therefore be factored out of the sum:

L_{\mathrm{obs}} =P(Y_1=1)q(1) =(0.20+0.40)(0.50) =(0.60)(0.50) =0.30.

The agreement confirms the simplification for this model and record. It does not imply that the same factorization is valid when the response probability depends on $Y_2$.

Justification and Likelihood Checks

Marginalization preserves the probability law because the complete histories compatible with distinct observed records partition the full-data possibilities. For each fixed complete history $y$, the response probabilities over all possible response patterns sum to one. Therefore, summing observed-data contributions across patterns and integrating over each pattern’s observed outcomes recovers total probability one:

\sum_r \int L_{\mathrm{obs}}(y_{\mathrm{obs}},r)\,d\nu(y_{\mathrm{obs}}) = \int f_\theta(y)\left\{\sum_r g_\psi(r\mid y)\right\}\,d\nu(y) = \int f_\theta(y)\,d\nu(y) =1.

The first equality rearranges sums and integrals over nonnegative probabilities or densities. The second uses the fact that $g_\psi(\cdot\mid y)$ is a probability distribution over response patterns for each $y$; the last uses normalization of the outcome distribution. This also gives practical checks: a proposed response model must assign valid probabilities to patterns, and the outcome model must be normalized on its support.

For $n$ independent participants with observed records $(y_{i,\mathrm{obs}},r_i)$, the sample likelihood is the product of their contributions:

L(\theta,\psi) = \prod_{i=1}^{n} L_{\mathrm{obs}}(\theta,\psi;y_{i,\mathrm{obs}},r_i).

Independence across participants is needed for this product form. Dependence within a participant’s repeated outcomes is already represented by the joint distribution $f_\theta(y)$; it is not appropriate to multiply marginal visit-specific outcome probabilities unless the model justifies that factorization.

Diagnostics, Alternatives, and Limitations

A reliable derivation makes clear which quantities are observed, which are integrated out, and what distribution is being modeled. Several errors are especially consequential:

Dropping the response pattern. Using only $f_\theta(y_{\mathrm{obs}})$ omits information in $r$ and generally does not give the joint observed-data likelihood contribution.
Removing a response factor that depends on missing outcomes. In the selection factorization, $g_\psi(r\mid y)$ remains inside the marginalization unless it is constant over the missing outcomes being summed or integrated out.
Using the wrong support. A sum must include every possible missing discrete value, and an integral must cover the full allowed range. Truncating either can produce a contribution that is too small.
Treating a missing outcome as a recorded value. Substituting a mean, zero, or a single imputed value is not the marginal likelihood; the contribution averages over the modeled possibilities with their appropriate probabilities or densities.
Confusing joint and conditional likelihoods. The contribution above includes the probability of the observed response pattern. Conditioning on that pattern changes the quantity being modeled and requires its own normalization.

The main alternative computational strategy is to model the observed-data distribution directly, without first writing a complete-data model. That can be convenient when the observed-data law is easy to specify, but it does not automatically provide a coherent model for the unseen outcomes. Complete-case analysis is simpler still, but it discards incomplete records and is not generally equivalent to maximizing the marginal likelihood. These approaches answer different modeling choices; none removes the need to state assumptions.

Marginalization is exact as a probability calculation, but it does not guarantee that the parameters are identifiable from the observed records. If different choices of the full outcome and response models produce the same observed-data distribution, the observed likelihood alone cannot distinguish them. Numerical integrations can also be difficult in high dimensions, and an incorrect or poorly fitting model for either factor can yield misleading inferences. The response mechanism and outcome model therefore need substantive justification and appropriate diagnostics; the observed-data likelihood is not a substitute for that assessment.

Common pitfall: “The missing outcome is unobserved, so its response probability can be ignored.” This is not generally correct. Under the selection factorization the response probability can depend on that outcome, and its value affects each term in the marginalization. The correction is to keep the response factor inside the sum or integral unless the model establishes that it is constant with respect to the missing outcome.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

For a discrete missing outcome with three possible values, what operation produces its observed-data likelihood contribution?
In the selection factorization, when can the response probability be taken outside the marginalization over a missing outcome?
Why is $f_\theta(y_{\mathrm{obs}})$ alone generally not the joint likelihood contribution for an incomplete record?
If $P(Y=0)=0.7$, $P(Y=1)=0.3$, $P(R=0\mid Y=0)=0.2$, and $P(R=0\mid Y=1)=0.6$, calculate the contribution for observing only $R=0$.

Solutions. (1) Sum the full-data joint contributions over all three possible values, multiplying each outcome probability or density by the response-pattern probability at that value. (2) It can be taken outside when it does not vary over the missing outcome values being summed or integrated, as when it depends only on observed history. (3) That expression omits the observed response pattern, whose probability is part of the joint observed record. (4) The contribution is $(0.7)(0.2)+(0.3)(0.6)=0.14+0.18=0.32$.