Marginalizing Missing Outcomes Within A Pattern

Why Marginalize Within a Pattern?

For a person assigned to a particular missingness pattern, some outcome components are observed and others are not. The pattern-specific outcome distribution describes the full outcome vector among people with that pattern, but the observed data contain only the components that were recorded. To obtain the density or probability of what was actually observed, we must sum or integrate over every possible value of the missing components. We retain the observed values; we do not substitute guessed values for the missing ones.

This operation is one step in constructing an observed-data likelihood. It is important to keep two quantities distinct: the distribution of observed outcomes conditional on the pattern, and the joint contribution that also includes the probability of being in that pattern. The distinction matters when comparing patterns or computing a likelihood for a sample.

Definition: Let $Y=(Y_1,\ldots,Y_K)$ be the full outcome vector, $R$ the response-pattern indicator, and $X=x$ fixed covariates. For a pattern $r$, let $O(r)$ be the indices of the observed outcomes and $M(r)$ the indices of the missing outcomes. Write $y_{O(r)}$ and $y_{M(r)}$ for the corresponding components. If $f_r(y\mid x)=f(Y=y\mid R=r,X=x)$ is the full-outcome density within pattern $r$, then the observed-outcome density within that pattern is $$ f_r(y_{O(r)}\mid x)=\int f_r(y_{O(r)},y_{M(r)}\mid x)\,dy_{M(r)}. $$ For discrete missing components, the integral is replaced by a sum. The joint observed-data contribution for a record in pattern $r$ is $P(R=r\mid X=x)f_r(y_{O(r)}\mid x)$.

The integration is over the support of the missing components. When there are several missing components, this is a multiple integral, and the observed components remain fixed at their recorded values throughout. If all outcomes are observed, there is nothing to integrate. If none are observed, the within-pattern observed-outcome contribution is the integral of the full density, which equals one; the joint contribution is then just the pattern probability.

Deriving the Observed-Outcome Density

The basic justification follows from the law of total probability. For a fixed observed value $y_{O(r)}$, all possible values of $Y_{M(r)}$ are mutually exclusive alternatives that can accompany it. Adding their probabilities, or integrating their joint density, gives the probability or density for the observed components alone.

\begin{aligned} f_r(y_{O(r)}\mid x) &=\int f_r(y_{O(r)},y_{M(r)}\mid x)\,dy_{M(r)}\\ &=\int f_r(y_{O(r)}\mid x) f_r(y_{M(r)}\mid y_{O(r)},x)\,dy_{M(r)}\\ &=f_r(y_{O(r)}\mid x) \int f_r(y_{M(r)}\mid y_{O(r)},x)\,dy_{M(r)}\\ &=f_r(y_{O(r)}\mid x). \end{aligned}

In this display, the first line is the marginalization definition. The second line uses the joint-density identity that factors the joint distribution into a marginal density and a conditional density. The conditional density integrates to one over the support of $Y_{M(r)}$, which gives the last line. This argument also explains why integrating a valid full-outcome density over its missing components produces a normalized marginal density: integrating the resulting marginal density over all values of $y_{O(r)}$ gives one.

If a pattern-specific full-outcome distribution is already expressed as a product of conditional densities, it can be useful to recognize which factors describe the missing components and integrate them. A conditional density integrates to one when its conditioning values are held fixed and the integration covers its full support. But a factor that depends on an observed value must not be discarded merely because it appears next to a missing-component factor.

Worked Marginalizations

Worked Example: Summing Over a Missing Binary Outcome

Suppose a hypothetical pattern $r$ records $Y_1$ but not $Y_2$, where both outcomes are binary. Within this pattern, suppose the full conditional probabilities are

$Y_1$	$Y_2$	$P(Y_1,Y_2\mid R=r)$
0	0	0.10
0	1	0.20
1	0	0.30
1	1	0.40

The four probabilities sum to $0.10+0.20+0.30+0.40=1.00$, so they form a valid full-outcome distribution within the pattern. For an individual with observed $Y_1=1$, marginalize over the two possible values of the missing $Y_2$:

P(Y_1=1\mid R=r) =\sum_{y_2\in\{0,1\}}P(Y_1=1,Y_2=y_2\mid R=r) =0.30+0.40=0.70.

Thus the within-pattern probability of the observed value is $0.70$. If, hypothetically, $P(R=r)=0.20$, the joint observed-data contribution is $0.20(0.70)=0.14$. The first quantity is conditional on pattern $r$; the second also accounts for the probability of that pattern.

Worked Example: Integrating a Missing Normal Outcome

Consider a hypothetical pattern $r$ in which $Y_1$ and $Y_3$ are observed but $Y_2$ is missing. Within this pattern, suppose $Y_1$, $Y_2$, and $\varepsilon$ are mutually independent, with $Y_1\sim N(0,1)$, $Y_2\sim N(0,1)$, and $\varepsilon\sim N(0,1)$, and let $Y_3=1+2Y_2+\varepsilon$. The conditional full-outcome density can be written

f_r(y_1,y_2,y_3) =\phi(y_1;0,1)\phi(y_2;0,1)\phi(y_3;1+2y_2,1),

where $\phi(y;\mu,\sigma^2)$ denotes the normal density at $y$ with mean $\mu$ and variance $\sigma^2$. Suppose the recorded values are $y_1=1$ and $y_3=2$. The observed-outcome density requires integrating out $y_2$:

\begin{aligned} f_r(1,2) &=\phi(1;0,1)\int_{-\infty}^{\infty} \phi(y_2;0,1)\phi(2;1+2y_2,1)\,dy_2\\ &=\phi(1;0,1)\phi(2;1,5)\\ &\approx 0.24197(0.16143)\\ &\approx 0.03906. \end{aligned}

To see the marginal normal distribution in the second line, $Y_3=1+2Y_2+\varepsilon$ has mean $1$ and variance $2^2(1)+1=5$, because $Y_2$ and $\varepsilon$ are independent. In particular, $\phi(2;1,5)=\exp(-1/10)/\sqrt{10\pi}\approx0.16143$. The value $0.03906$ is a density for the two observed outcomes, not a probability at a point. If $P(R=r)=0.25$, the joint contribution is $0.25(0.03906)\approx0.009766$.

Worked Example: Taking the Marginal Subvector of a Multivariate Normal

Suppose the full outcome vector within a hypothetical pattern $r$ has a multivariate normal distribution with mean $(10,20,30)$ and covariance matrix

\Sigma= \begin{pmatrix} 4&1&0.5\\ 1&9&2\\ 0.5&2&16 \end{pmatrix}.

Suppose $Y_1$ and $Y_3$ are observed and $Y_2$ is missing, with recorded values $Y_1=12$ and $Y_3=26$. A marginal subvector of a multivariate normal distribution is normal: its mean and covariance are the corresponding entries of the full mean and covariance. Therefore $(Y_1,Y_3)$ has mean $(10,30)$ and covariance matrix

\Sigma_O= \begin{pmatrix} 4&0.5\\ 0.5&16 \end{pmatrix}.

The determinant is $4(16)-0.5^2=63.75$, and the inverse is $\Sigma_O^{-1}=\frac{1}{63.75}\begin{pmatrix}16&-0.5\\-0.5&4\end{pmatrix}$. The centered observed vector is $d=(2,-4)^\mathsf{T}$. Its quadratic form is

d^\mathsf{T}\Sigma_O^{-1}d =\frac{16(2^2)+2(-0.5)(2)(-4)+4((-4)^2)}{63.75} =\frac{136}{63.75} \approx2.1333.

The bivariate normal density at the observed values is consequently

\begin{aligned} f_r(12,26) &=\frac{\exp\{-\tfrac12(2.1333)\}} {2\pi\sqrt{63.75}}\\ &\approx 0.006859. \end{aligned}

This is the result of integrating the full three-outcome density over $Y_2$. If $P(R=r)=0.30$, the joint observed-data contribution is approximately $0.30(0.006859)=0.002058$. Notice that marginalization uses the covariance submatrix for the observed outcomes. It does not replace the missing outcome by its conditional mean given the observed values.

Assumptions, Checks, and Failure Modes

Marginalization is an identity, but its use in a likelihood calculation still depends on having a valid model and the right pattern specification. Check that the full-outcome distribution within the pattern is normalized, that the integration covers the support of every missing component, and that the observed values are held fixed. For a continuous outcome, the result is a density with units inverse to those of the observed components; for discrete outcomes, it is a probability mass. The pattern probability is dimensionless and multiplies the conditional density or mass.

A useful internal check is to integrate the resulting observed-outcome density over all possible observed values. It must equal one within the pattern. Also check that pattern probabilities are handled once: they are not part of $f_r(y\mid x)$ if that notation denotes a distribution conditional on $R=r$, but they are part of the joint observed-data contribution.

Substituting a single missing value. Replacing a missing outcome by its mean, mode, or another guessed value evaluates the full density at one point; it does not sum or integrate over all possible values.
Dropping a factor too early. A factor involving an observed component can remain after integrating a missing component. In the normal example, $\phi(1;0,1)$ remains because $Y_1=1$ is observed.
Confusing conditional and joint contributions. Multiplying by $P(R=r\mid x)$ is required for a joint observed-data contribution, but not for the density conditional on $R=r$.
Changing a marginal into a conditional distribution. Marginalizing $Y_2$ gives the distribution of the observed components. It is not the distribution of $Y_2$ conditional on those components.

The calculation also requires attention to the modeled distribution. If the assumed pattern-specific full-outcome density is wrong, integrating it exactly still gives the wrong observed-data density. Further, data from pattern $r$ directly reveal the distribution of its observed components, but not the values of its missing components. The model for those missing components is therefore not checked by simply evaluating observed records in that pattern.

How This Marginalization Fits with Other Approaches

The pattern-mixture approach starts with outcome distributions conditional on missingness patterns. Within a pattern, marginalization reduces that pattern’s full-outcome distribution to the components actually recorded. The selection factorization instead represents the joint law using an outcome distribution and a response distribution conditional on outcomes; deriving observed-data contributions there also requires summing or integrating over unobserved outcomes. These are different factorizations of a joint law, not different rules for marginalization.

A third practical option is to specify the observed-outcome marginal distribution directly for each pattern, without first writing a full-outcome density. That can be convenient when only the observed-data likelihood is needed. The direct specification must nevertheless be a valid normalized distribution and be compatible with whatever full-data model or interpretation is intended. Marginalization from a full pattern-specific model makes the relationship explicit, at the cost of requiring a model for components that are missing in that pattern.

Marginalization does not, by itself, establish that a chosen full-data model is appropriate or that its parameters can all be learned from observed records. Its immediate purpose is narrower: calculate the probability or density of the observed components under the specified model. The distinction between a valid calculation and a justified modeling assumption is essential.

Key takeaway: For a fixed pattern, retain the observed outcomes and integrate or sum the full pattern-specific density over every missing outcome. Multiply the resulting conditional observed-outcome density by the pattern probability only when forming the joint observed-data contribution.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a pattern, $Y_1$ is observed and binary while $Y_2$ is missing. If $P(Y_1=0,Y_2=0\mid r)=0.15$ and $P(Y_1=0,Y_2=1\mid r)=0.25$, what is the within-pattern probability of observing $Y_1=0$?
If that pattern has probability $0.40$, what is the joint observed-data contribution for the observation in Question 1?
For $Z=3+W+\varepsilon$, where $W\sim N(0,4)$ and $\varepsilon\sim N(0,1)$ are independent, what are the marginal mean and variance of $Z$?
Why is evaluating a full density at the mean of a missing outcome generally not the same as marginalizing that outcome?

Solutions. (1) Sum over both values of the missing binary outcome: $P(Y_1=0\mid r)=0.15+0.25=0.40$. (2) Multiply by the pattern probability: $0.40(0.40)=0.16$. (3) The mean is $3+E(W)+E(\varepsilon)=3$, and independence gives variance $4+1=5$, so $Z\sim N(3,5)$. (4) Marginalization sums or integrates the joint density over the full support of the missing outcome. Evaluation at one value uses only one point and omits the contributions from all other possible values.

\(Y_1\)	\(Y_2\)	\(P(Y_1,Y_2\mid R=r)\)
0	0	0.10
0	1	0.20
1	0	0.30
1	1	0.40