Why Condition on the Blocks That Are Observed?
When one or more omics modalities are absent for a subject, the available blocks can still contain information about the missing measurements. Under a joint Gaussian model, that information has a precise form: the conditional distribution of the missing block given the observed blocks. Its mean gives a model-based prediction, while its covariance describes the remaining uncertainty after conditioning.
This tutorial focuses on that conditional distribution. It is not enough to compute a predicted value and treat it as observed without qualification. The prediction depends on the Gaussian model and its covariance structure; the conditional covariance is therefore part of the result, not an optional extra.
The formula applies to blocks of any finite dimension. For example, \(Z_o\) may collect measurements from two observed modalities, while \(Z_m\) contains several features from a modality that is absent for the subject. The same block formula applies whether the missing block has one feature or many.
Deriving the Conditional Distribution
Assume first that the full vector is jointly Gaussian and that \(\Sigma_{oo}\) is positive definite. Define the matrix \(B=\Sigma_{mo}\Sigma_{oo}^{-1}\), and form the residual $$ R=Z_m-\mu_m-B(Z_o-\mu_o). $$ Its covariance with the observed block is $$ \operatorname{Cov}(R,Z_o) =\Sigma_{mo}-B\Sigma_{oo} =\Sigma_{mo}-\Sigma_{mo}\Sigma_{oo}^{-1}\Sigma_{oo} =0. $$ Because \(R\) and \(Z_o\) are jointly Gaussian, zero cross-covariance implies that they are independent. Thus, after observing \(Z_o=z_o\), the distribution of \(R\) is unchanged. The missing block is \(Z_m=\mu_m+B(z_o-\mu_o)+R\), so its conditional mean is \(\mu_m+B(z_o-\mu_o)\).
The covariance of the residual, and hence the conditional covariance, is $$ \begin{aligned} \operatorname{Var}(R) &=\operatorname{Var}(Z_m-BZ_o)\\ &=\Sigma_{mm}-B\Sigma_{om}-\Sigma_{mo}B^T+B\Sigma_{oo}B^T\\ &=\Sigma_{mm}-\Sigma_{mo}\Sigma_{oo}^{-1}\Sigma_{om}. \end{aligned} $$ The last equality uses \(B=\Sigma_{mo}\Sigma_{oo}^{-1}\) and the symmetry of \(\Sigma_{oo}^{-1}\). This establishes both parts of the conditional distribution, rather than only its prediction formula.
The conditional covariance is positive semidefinite. In particular, for any vector \(v\), the amount subtracted from the marginal variance in direction \(v\) is $$ v^T\Sigma_{mo}\Sigma_{oo}^{-1}\Sigma_{om}v =(\Sigma_{om}v)^T\Sigma_{oo}^{-1}(\Sigma_{om}v)\geq 0, $$ because \(\Sigma_{oo}^{-1}\) is positive definite. Conditioning therefore cannot increase the model-based variance of a linear combination of the missing block. It can leave the variance unchanged when the observed and missing blocks have no linear association under the model.
Worked Example: One Missing Feature and One Observed Feature
Suppose a hypothetical two-feature omics vector has observed feature \(A\) and missing feature \(B\). Under a bivariate Gaussian model, their means are \(\mu_A=10\) and \(\mu_B=20\), their variances are \(4\) and \(9\), and their covariance is \(3\). For a subject with \(A=12\), find the conditional distribution of \(B\).
The observed variance is positive, so the required inverse exists. The joint covariance is also positive definite: its determinant is \(4(9)-3^2=27>0\). The conditional mean and variance are $$ \begin{aligned} E(B\mid A=12) &=20+\frac{3}{4}(12-10) =20+1.5 =21.5,\\ \operatorname{Var}(B\mid A=12) &=9-\frac{3^2}{4} =9-2.25 =6.75. \end{aligned} $$ Thus \(B\mid A=12\sim N(21.5,6.75)\). The conditional standard deviation is \(\sqrt{6.75}\approx2.598\) feature units. Filling the missing value with 21.5 is a conditional-mean imputation; it does not mean that the unknown measurement is known to equal 21.5.
Multivariate Blocks and Worked Calculations
With several observed features, the conditional mean uses a matrix of coefficients, and the conditional covariance describes uncertainty and dependence across the missing features. Matrix dimensions are a useful check: if \(Z_o\) has \(q\) entries and \(Z_m\) has \(r\), then \(\Sigma_{mo}\) is \(r\) by \(q\), and the conditional covariance is \(r\) by \(r\).
Worked Example: Two Observed Features Predict One Missing Feature
Consider a hypothetical three-feature vector. Two observed features have mean vector \(\mu_o=(10,30)^T\) and covariance matrix $$ \Sigma_{oo}= \begin{pmatrix} 4&1\\ 1&9 \end{pmatrix}. $$ The missing feature has mean \(5\) and variance \(16\), with cross-covariance row \(\Sigma_{mo}=(2,3)\). The observed values are \(z_o=(11,28)^T\). Compute its conditional mean and variance.
The determinant of \(\Sigma_{oo}\) is \(4(9)-1=35>0\), and its first diagonal entry is positive, so \(\Sigma_{oo}\) is positive definite. Its inverse and the conditional-mean coefficient row are $$ \Sigma_{oo}^{-1} =\frac{1}{35} \begin{pmatrix} 9&-1\\ -1&4 \end{pmatrix}, \qquad \Sigma_{mo}\Sigma_{oo}^{-1} =\left(\frac{3}{7},\frac{2}{7}\right). $$ The observed deviation from its mean is \((1,-2)^T\). Therefore, $$ E(Z_m\mid Z_o=z_o) =5+\left(\frac{3}{7},\frac{2}{7}\right) \begin{pmatrix}1\\-2\end{pmatrix} =5-\frac{1}{7} =\frac{34}{7} \approx4.857. $$ For the conditional variance, the reduction from the marginal variance is $$ (2,3)\Sigma_{oo}^{-1} \begin{pmatrix}2\\3\end{pmatrix} =\left(\frac{3}{7},\frac{2}{7}\right) \begin{pmatrix}2\\3\end{pmatrix} =\frac{12}{7}. $$ Hence \(\operatorname{Var}(Z_m\mid Z_o=z_o)=16-\frac{12}{7}=\frac{100}{7}\approx14.29\) feature-units squared. The conditional standard deviation is approximately \(3.780\) feature units. The two observed features jointly shift the prediction; neither the prediction nor its uncertainty can generally be found by treating them as unrelated one-at-a-time inputs.
Worked Example: A Missing Two-Feature Block
Suppose one observed feature \(A\) has mean \(1\) and variance \(4\). Two missing features \(B,C\) have mean vector \((2,3)^T\), covariance matrix $$ \Sigma_{mm}= \begin{pmatrix} 5&1\\ 1&3 \end{pmatrix}, \qquad \Sigma_{mo}= \begin{pmatrix} 2\\ -1 \end{pmatrix}. $$ For a hypothetical subject with \(A=3\), calculate the conditional mean vector and covariance matrix for \((B,C)^T\).
The observed deviation is \(3-1=2\). The conditional mean is $$ \begin{pmatrix}2\\3\end{pmatrix} +\begin{pmatrix}2\\-1\end{pmatrix}\frac{1}{4}(2) = \begin{pmatrix}2\\3\end{pmatrix} +\begin{pmatrix}1\\-0.5\end{pmatrix} = \begin{pmatrix}3\\2.5\end{pmatrix}. $$ The covariance reduction and resulting conditional covariance are $$ \begin{pmatrix}2\\-1\end{pmatrix} \frac{1}{4} \begin{pmatrix}2&-1\end{pmatrix} = \begin{pmatrix}1&-0.5\\-0.5&0.25\end{pmatrix}, \qquad \operatorname{Var}\left( \begin{pmatrix}B\\C\end{pmatrix}\middle|A=3 \right) = \begin{pmatrix}4&1.5\\1.5&2.75\end{pmatrix}. $$ This covariance is positive definite: its determinant is \(4(2.75)-1.5^2=8.75>0\), and its first diagonal entry is positive. The conditional variances are 4 and 2.75 feature-units squared, and the conditional covariance is 1.5 feature-units squared. Thus the two missing features remain dependent after conditioning on \(A\); replacing both by their conditional means would not represent that residual dependence.
Assumptions, Diagnostics, and Alternatives
The formula describes a missing block under a joint Gaussian model for the relevant features. Its parameters may be specified or estimated; when estimated, the plug-in conditional distribution treats those estimates as fixed and does not automatically include uncertainty from estimating the mean and covariance. The observed-block covariance must be invertible. In practice, a covariance estimate that is singular or nearly singular can make the inverse unavailable or unstable, especially when the observed block has many features relative to the number of subjects.
The missingness mechanism also matters for interpreting the result. If modality availability is ignorable under the conditions described in the earlier material on MCAR, MAR, and MNAR, the conditional distribution can be used as a model for the missing measurements given the observed measurements. Under informative, nonignorable missingness, the distribution among subjects with an absent block may differ from the ordinary Gaussian conditional distribution; a model for the missingness process or a sensitivity analysis may then be needed.
Useful checks include verifying that the covariance blocks have matching dimensions, that \(\Sigma_{oo}\) is symmetric positive definite, and that the resulting conditional covariance is symmetric positive semidefinite. Also compare model-implied conditional predictions with observed values for subjects whose corresponding blocks are available. Large systematic prediction errors, implausible conditional variances, or residual patterns across modality-availability groups can signal poor fit or an inappropriate covariance model. Such checks cannot prove that the Gaussian assumptions hold for truly unobserved values.
Conditional-mean imputation is useful when a single predicted value is needed, but it suppresses residual variation. For squared-error loss, the conditional mean minimizes expected squared prediction error given the observed block; this optimality does not make the filled-in values equivalent to measurements. Complete-case analysis avoids filling in values but discards subjects without every required modality and may target a selected subset. Marginal-mean imputation ignores the observed cross-modal information, while a non-Gaussian predictive model can be more suitable when the Gaussian shape or linear conditional mean is implausible. These alternatives answer different practical needs; none removes the need to state the target and assumptions.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- If the cross-covariance between observed and missing blocks is zero, what are the conditional mean and covariance?
- A scalar missing feature has variance 10, an observed feature has variance 5, and their covariance is 2. What is the conditional variance?
- Why does a conditional covariance matrix describe more than the separate uncertainties of each missing feature?
- What does a conditional-mean imputation fail to preserve when it is treated as an exact measurement?
Solutions. (1) The conditional mean is \(\mu_m\), and the conditional covariance is \(\Sigma_{mm}\), because both terms involving \(\Sigma_{mo}\) are zero. (2) The conditional variance is \(10-2^2/5=10-0.8=9.2\), in squared feature units. (3) Its off-diagonal entries describe conditional covariances between missing features, which represent their remaining dependence. (4) It fails to represent residual variation and uncertainty around the prediction, and can distort variability and dependence in subsequent analyses.