Observed-Data Likelihood for Partially Observed Blocks

Why the Likelihood Must Follow the Observation Pattern

In a multi-omics study, one subject may have measurements from every modality, while another has only one block available. A joint model describes the full vector of measurements, but a likelihood for the recorded data must describe only what was actually observed. The missing coordinates are not known values to insert into a calculation: they must be integrated out of the joint model.

This distinction matters because different subjects can contribute different-dimensional densities. If a subject has only modality A recorded, that subject contributes the marginal density for A, not a joint density that pretends the unobserved modalities were measured. If the joint model is coherent, these pattern-specific contributions can be multiplied to form one observed-data likelihood.

Definition: Let $Y_i$ be the full vector of measurements for subject $i$, and let $O_i$ be the set of coordinates observed for that subject. Write $y_{i,O_i}$ for the observed values. Under a specified joint density $f(y_i;\theta)$, the observed-data density for subject $i$ is $$ f_{O_i}(y_{i,O_i};\theta) = \int f(y_{i,O_i},y_{i,O_i^c};\theta)\,dy_{i,O_i^c}, $$ where $O_i^c$ is the set of unobserved coordinates. This is the marginal density of the observed coordinates under the joint model.

The likelihood is the observed-data density viewed as a function of the unknown model parameter $\theta$, with the recorded measurements held fixed. If subjects are independent, the outcome-model likelihood is

L(\theta) = \prod_{i=1}^{n} f_{O_i}(y_{i,O_i};\theta), \qquad \ell(\theta) = \sum_{i=1}^{n}\log f_{O_i}(y_{i,O_i};\theta).

Independence here is a modeling assumption about subjects, not about modalities within a subject. The purpose of the joint model is precisely to allow dependence between modalities. In applications, the model may be conditional on covariates; then the densities above are interpreted as conditional on those covariates.

Gaussian Marginal Contributions

Suppose the full measurement vector follows a multivariate normal distribution, $Y_i\sim N(\mu,\Sigma)$, where $\mu$ is a mean vector and $\Sigma$ is a covariance matrix. Assume $\Sigma$ is positive definite, so the full vector has a nonsingular density. For any observed coordinate set $O_i$, the marginal vector is also multivariate normal, with mean vector $\mu_{O_i}$ and covariance matrix $\Sigma_{O_iO_i}$. These are the entries of the full mean and covariance restricted to the observed coordinates.

Gaussian observed-data contribution: If $d_i$ coordinates are observed for subject $i$, then $$ f_{O_i}(y_{i,O_i};\theta) = \frac{ \exp\!\left[ -\frac{1}{2} (y_{i,O_i}-\mu_{O_i})^\mathsf{T} \Sigma_{O_iO_i}^{-1} (y_{i,O_i}-\mu_{O_i}) \right] }{ (2\pi)^{d_i/2}\, |\Sigma_{O_iO_i}|^{1/2} }. $$ The covariance submatrix must be positive definite for this density formula to apply.

The determinant and inverse in the expression are those of the observed-coordinate covariance submatrix, not of the full covariance matrix. Consequently, the contribution changes with the availability pattern. A subject with only one coordinate contributes a univariate normal density; a subject with two observed coordinates contributes a bivariate marginal density.

Worked Example: One Observed Modality From a Bivariate Gaussian Model

Suppose two standardized modalities $A$ and $B$ have the specified joint model $(A,B)^\mathsf{T}\sim N((0,0)^\mathsf{T},\Sigma)$, with $\Sigma=\begin{pmatrix}1&0.5\\0.5&1\end{pmatrix}$. A subject has $A=1$, while $B$ is unobserved. Find this subject’s likelihood contribution.

Check the model condition. The covariance matrix has determinant $1-(0.5)^2=0.75>0$, and its diagonal entries are positive, so it is positive definite. Its marginal variance for $A$ is 1. The fact that $B$ is missing does not change that marginal variance.

Marginalize over $B$. The distribution of $A$ is $N(0,1)$, so the contribution is $$ f_A(1) = \frac{1}{\sqrt{2\pi}} \exp\!\left(-\frac{1^2}{2}\right) = \frac{0.60653}{2.50663} \approx 0.2420. $$

This is a density in the standardized units of $A$, not a probability that $A$ equals exactly 1. The correlation with $B$ is part of the joint model, but it does not enter this marginal density when $B$ is unobserved and the marginal variance of $A$ is fixed at 1.

Combining Different Patterns in One Likelihood

A coherent joint model lets the likelihood use every subject’s observed measurements without requiring every subject to have the same pattern. The next calculation combines univariate and bivariate contributions under one specified model.

Worked Example: Three Subjects With Two Availability Patterns

Continue with the model in the previous example: $A$ and $B$ have zero means, unit variances, and correlation $0.5$. Consider three independent subjects: subject 1 has $A=1$ only; subject 2 has $A=1$ and $B=-1$; subject 3 has $B=0$ only. Compute the likelihood at the specified parameter values.

Subject 1. As above, the $A$-only contribution is $0.24197$. Subject 3 has $B=0$ only, and the marginal distribution of $B$ is also $N(0,1)$, so $$ f_B(0)=\frac{1}{\sqrt{2\pi}}\exp(0)\approx 0.39894. $$

Subject 2. For the bivariate observation $(1,-1)$, the determinant of $\Sigma$ is $0.75$, and the quadratic form is $$ (1,-1)\Sigma^{-1}(1,-1)^\mathsf{T} = \frac{1^2+(-1)^2-2(0.5)(1)(-1)}{0.75} = \frac{3}{0.75} =4. $$ Thus the bivariate density is $$ f_{A,B}(1,-1) = \frac{\exp(-4/2)}{2\pi\sqrt{0.75}} = \frac{0.13534}{5.44140} \approx 0.02487. $$

Combine the contributions. Independence across subjects gives $$ L = f_A(1)\,f_{A,B}(1,-1)\,f_B(0) \approx (0.24197)(0.02487)(0.39894) \approx 0.002400. $$ This product is the likelihood at the specified means, variances, and correlation. It is not a probability for the three exact continuous observations.

For parameter estimation, one would evaluate this likelihood over candidate parameter values and seek values that make the observed data relatively more likely. A numerical likelihood value by itself is not an absolute measure of model adequacy: its scale depends on the measurement units and on the dimension of each density contribution.

Why Integration Produces the Correct Marginal

The operation in the definition is ordinary probability marginalization. For two blocks $A$ and $B$, the joint density can be factored as $f_{A,B}(a,b)=f_A(a)f_{B\mid A}(b\mid a)$. Integrating over the unobserved $b$ gives

\int f_{A,B}(a,b)\,db = f_A(a)\int f_{B\mid A}(b\mid a)\,db = f_A(a)\cdot 1 = f_A(a).

The conditional density integrates to one for every $a$, which is why the missing coordinate disappears and leaves the marginal density. In a multivariate normal model, a subvector of a jointly normal vector is itself normal; its mean and covariance are the corresponding subvector and principal submatrix. Applying that result to each pattern yields the Gaussian formula above. Crucially, the cross-covariances still matter for patterns that observe multiple modalities, even though they do not appear in a single-modality marginal.

Worked Example: A Two-Coordinate Marginal From a Three-Modality Model

Suppose $(A,B,C)^\mathsf{T}\sim N(0,\Sigma)$, where $\Sigma=\begin{pmatrix}1&0&0.3\\0&2&0\\0.3&0&4\end{pmatrix}$. A subject has $A=1$ and $C=2$, while $B$ is unobserved. Find the observed-data density contribution.

Check positive definiteness. The $A,C$ block has determinant $1(4)-(0.3)^2=3.91>0$; the independent $B$ variance is 2. Thus the full covariance matrix is positive definite. The observed covariance submatrix is $\Sigma_{AC}=\begin{pmatrix}1&0.3\\0.3&4\end{pmatrix}$, also with determinant 3.91.

Use the marginal model. The observed pair has mean zero. Its quadratic form is $$ \begin{pmatrix}1&2\end{pmatrix} \Sigma_{AC}^{-1} \begin{pmatrix}1\\2\end{pmatrix} = \frac{4(1)^2-2(0.3)(1)(2)+(2)^2}{3.91} = \frac{6.8}{3.91} \approx 1.7391. $$ Therefore $$ f_{A,C}(1,2) = \frac{\exp(-1.7391/2)}{2\pi\sqrt{3.91}} \approx \frac{0.4191}{12.4242} \approx 0.03374. $$ The subject contributes this bivariate marginal density. No value for $B$ is imputed or needed in the likelihood calculation.

When Can the Availability Process Be Left Out?

The product of outcome-model marginals is the appropriate likelihood for the measurement parameters when the availability mechanism is ignorable for that inference. A standard sufficient setup is that the missingness is missing at random (MAR), the parameters governing availability are distinct from the outcome-model parameters, and the model and sampling assumptions otherwise hold. Under MAR, the probability of an observed availability pattern may depend on observed data but not on the unobserved values once the observed data are fixed. Missing completely at random (MCAR) is a special case.

If availability depends on unobserved measurements even after conditioning on observed data, it is nonignorable for the outcome model. Then an outcome likelihood alone may not describe the observed-data process. For example, if $R_i$ denotes the availability pattern and $g(R_i\mid Y_i;\psi)$ models its dependence on the full measurement vector, the joint observed-data contribution has the form

\int f(y_{i,O_i},y_{i,O_i^c};\theta)\, g(R_i\mid y_{i,O_i},y_{i,O_i^c};\psi) \,dy_{i,O_i^c}.

This expression includes the availability mechanism rather than treating it as irrelevant. Which likelihood is justified therefore depends on the study design and missingness assumptions, not only on the algebra of Gaussian marginalization. A structurally unavailable modality also requires care: the scientific model and target population must clarify whether that modality is a missing value of a defined variable or was not defined or collected for that subject at all.

Diagnostics, Alternatives, and Limitations

Before optimizing an observed-data likelihood, check that the specified joint model is meaningful for every pattern. In the Gaussian case, the full covariance must be symmetric and positive definite for the stated nonsingular density, and the corresponding observed submatrices must also be positive definite. A set of separately estimated pairwise covariances may fail to form a coherent positive-definite matrix, as illustrated by the earlier material on pairwise covariances. A joint likelihood avoids that inconsistency by requiring one shared covariance model, but it does not guarantee that the model fits the data.

Inspect patterns and sample sizes. Tabulate which coordinates are observed together. Sparse patterns can leave covariance parameters weakly informed, even when the likelihood can be written down.
Check the fitted covariance. A non-positive-definite estimate, or an optimizer repeatedly approaching a boundary, can indicate an unsuitable parameterization, inadequate information, or model misspecification.
Assess the distributional model. Strong skewness, heavy tails, outliers, or nonlinear dependence can make Gaussian marginal contributions poor descriptions of the observed blocks.
Examine missingness assumptions. A plausible marginal model does not establish MAR. Sensitivity to departures from ignorability may be needed when nonignorable missingness is scientifically credible.

Complete-case analysis uses only subjects with all required modalities observed. It is simpler, but discards partial blocks and may change the target or introduce bias depending on selection. Pairwise methods use available pairs for separate summaries or covariance estimates, but those estimates need not be jointly coherent. The observed-data likelihood instead uses a specified joint model and each subject’s appropriate marginal. Its advantage is coherent use of partial observations; its limitations are dependence on the model, assumptions about missingness, and the information available across patterns. It is not itself an imputation procedure or a guarantee of identifiability.

Common pitfall: Treating a missing modality as zero, or evaluating a full joint density after filling in an invented value, does not marginalize over the unobserved measurement. The correction is to integrate the joint density over that coordinate, which gives the marginal density for the observed block. Also, a density such as $f_A(1)$ is not the probability that a continuous measurement equals 1.

Check Your Understanding

Answer each question using the definitions and assumptions in this tutorial. Short solutions follow the questions.

Under a specified joint model for $A$ and $B$, what density contributes when a subject has $A=a$ observed and $B$ missing?
For independent subjects with different observed patterns, how are their outcome-model likelihood contributions combined?
In a multivariate normal model, which covariance matrix is used for a subject who has only a subset of coordinates observed?
Why is it not generally valid to ignore the availability mechanism when missingness depends on unobserved measurements?
For a standard normal variable, calculate the contribution from one observed value $x=0$, and state what the density does and does not mean.

Solutions. (1) It is the marginal density $f_A(a)=\int f_{A,B}(a,b)\,db$. (2) Multiply the pattern-specific marginal densities, one for each independent subject, or add their log densities. (3) Use the principal covariance submatrix corresponding to the observed coordinates, together with the corresponding mean subvector. (4) The pattern then contains information about unobserved outcomes, so the observed-data process generally depends on the missingness mechanism as well as the outcome model. (5) $f(0)=(2\pi)^{-1/2}\exp(0)\approx0.3989$. It is a density per unit of measurement, not the probability that a continuous variable equals exactly zero.