Why Impute Whole Blocks Jointly?
When an individual is missing an entire omics modality, the unobserved values are a vector, not a collection of unrelated blanks. A defensible imputation should represent both the uncertainty in that vector and its conditional relationship to the modalities and covariates that were observed for that individual. Joint multiple imputation does this by specifying one joint model, drawing model parameters, and then drawing each missing block from its conditional distribution.
The central distinction is between a single completed data set and a set of plausible completed data sets. A conditional mean prediction fills in a best-guess value but leaves out the variation of plausible values around that mean. Multiple imputation instead generates multiple completed data sets, each reflecting uncertainty about both the missing values and, when the procedure is properly constructed, the model parameters. Analyses are carried out separately on the completed data sets; how to combine those analyses is a separate topic.
Assumptions and Model Specification
A common starting point for continuous, appropriately transformed and scaled measurements is a multivariate Gaussian model: \(Y_i\mid C_i \sim N\{\mu(C_i;\theta),\Sigma(\theta)\}\). Here \(\mu(C_i;\theta)\) describes the conditional mean vector and \(\Sigma(\theta)\) describes residual variances and covariances. The mean may include covariate effects; the covariance must be a valid positive-definite covariance matrix for the modeled variables. The model should be specified for the variables whose values are meaningfully unobserved, not for a modality that is structurally undefined for a subject.
The missingness assumption matters. Under missing at random (MAR), the probability of a subject’s availability pattern, conditional on observed measurements and covariates, does not additionally depend on the missing measurements. In notation, if \(R_i\) records availability, MAR means \(P(R_i\mid Y_{i,\mathrm{obs}},Y_{i,\mathrm{mis}},C_i)=P(R_i\mid Y_{i,\mathrm{obs}},C_i)\). For likelihood- or Bayesian-based imputation, the missingness mechanism can then be ignored when estimating the measurement model under the usual distinctness condition: parameters governing missingness are separate from parameters governing the data model. These conditions do not make MAR testable from observed data alone. If missingness depends on the unseen values even after conditioning on observed data, an ordinary MAR imputation does not resolve that dependence.
There must also be information to estimate the relationships used for prediction. If a target modality is never observed, or is observed only in a very small and unrepresentative set of subjects, its relationship to other modalities may not be identified reliably. Availability patterns and overlap therefore inform whether a joint model is estimable; a model cannot manufacture empirical evidence absent from the observed data.
Conditional Distribution for a Missing Block
Partition a subject’s Gaussian measurement vector into observed components \(Y_o\) and missing components \(Y_m\). Conditional on the covariates, write its mean and covariance as \(\mu=(\mu_o^\mathsf{T},\mu_m^\mathsf{T})^\mathsf{T}\) and \(\Sigma=\begin{pmatrix}\Sigma_{oo}&\Sigma_{om}\\\Sigma_{mo}&\Sigma_{mm}\end{pmatrix}\). Provided \(\Sigma_{oo}\) is invertible, the conditional distribution for the missing block is
The conditional mean adjusts the missing-block mean according to how the observed values depart from their own mean. The conditional covariance is the remaining uncertainty after using the observed block. It includes covariance among the missing features, so the draw preserves their modeled joint variation rather than sampling each feature independently.
From the Conditional Model to Multiple Imputations
A proper joint procedure does not hold all fitted parameters fixed. For imputation \(l\), first draw \(\theta^{(l)}\) from an uncertainty distribution based on the observed data, such as a posterior distribution under a specified Bayesian model or a suitable frequentist resampling or asymptotic approximation. Then, for each subject with missing measurements, draw the entire missing vector from the conditional Gaussian distribution evaluated at \(\theta^{(l)}\). The observed values remain unchanged. Repeating this process yields multiple completed data sets.
There are two sources of variation in those draws. The conditional covariance represents uncertainty about a subject’s missing values given the model parameters. Variation in \(\theta^{(l)}\) represents uncertainty about the fitted model itself. Drawing residuals while fixing parameters at one estimate captures the first source, but generally omits the second. That omission can make imputed data sets too similar and downstream uncertainty too small.
Specify the measurement distribution, covariate effects, and covariance structure for the continuous variables to be imputed. Check that the assumptions are plausible for the transformed data.
Fit the model using observed information and obtain a valid draw of its parameters for each imputation. In particular, covariance draws must remain valid covariance matrices.
For each availability pattern and subject, form the conditional distribution given that subject’s observed modalities and covariates, then sample the missing vector jointly.
Generate several completed data sets with distinct parameter and predictive draws. Inspect convergence or sampling quality where relevant, and assess whether the model and imputations are plausible.
Worked Example: Drawing a Two-Feature Missing Block
Suppose a hypothetical subject has two missing standardized omics measurements, \(P\) and \(M\), and the joint model gives their conditional mean, given the subject’s observed modalities and covariates, as \((1.2,-0.5)^\mathsf{T}\). For this illustrative draw, suppose the conditional covariance is \(\begin{pmatrix}3.75&0.5\\0.5&8\end{pmatrix}\). Assume these values are from one parameter draw; the model also allows parameter values to vary across imputations.
To sample the block, factor the covariance as \(\Sigma=LL^\mathsf{T}\), where \(L=\begin{pmatrix}\sqrt{3.75}&0\\0.5/\sqrt{3.75}&\sqrt{8-0.5^2/3.75}\end{pmatrix}\). Numerically, \(L\approx\begin{pmatrix}1.9365&0\\0.2582&2.8166\end{pmatrix}\). The diagonal entry in the second row is \(\sqrt{8-0.06667}=\sqrt{7.93333}\approx2.8166\); this makes the second row’s squared length \(0.5^2/3.75+2.8166^2\approx8\), as required.
Use the illustrative standard-normal draw \(z=(0.4,-0.2)^\mathsf{T}\). The deviation from the conditional mean is \(Lz\approx(1.9365(0.4),\ 0.2582(0.4)+2.8166(-0.2))^\mathsf{T}\approx(0.7746,-0.4600)^\mathsf{T}\). Therefore the imputed block is \((P,M)^\mathsf{T}\approx(1.2,-0.5)^\mathsf{T}+(0.7746,-0.4600)^\mathsf{T}=(1.9746,-0.9600)^\mathsf{T}\). This is one plausible draw, not a claim that these are the subject’s known values. The off-diagonal covariance affects the joint draw; it should not be replaced by two unrelated marginal draws.
Worked Example: Why Draw Model Parameters?
Consider a deliberately simple hypothetical conditional model for one missing measurement \(Y\). Given an unknown mean parameter \(\theta\), let \(Y\mid\theta\sim N(\theta,1)\). Suppose the observed data imply the parameter distribution \(\theta\mid\text{observed data}\sim N(2,0.25)\). For one imputation, draw \(\theta=2.3\), then draw a residual using the standard-normal value \(-0.5\).
The imputed value is \(Y=2.3+1(-0.5)=1.8\). If instead the parameter were fixed at its estimated mean \(2\), the same residual draw would give \(Y=2-0.5=1.5\). These are different imputations because the parameter draw contributes variation in addition to the subject-level residual.
The predictive variance in this simple model illustrates the two components. By the law of total variance, \(\operatorname{Var}(Y\mid\text{observed data})=\mathbb{E}\{\operatorname{Var}(Y\mid\theta)\}+\operatorname{Var}\{\mathbb{E}(Y\mid\theta)\}=1+0.25=1.25\). Fixing \(\theta=2\) retains variance \(1\) but omits the additional \(0.25\) from uncertainty about the mean.
Worked Example: Sampling a Block Instead of Separate Features
For another hypothetical subject, suppose the conditional mean of a missing two-feature block is \((4,1)^\mathsf{T}\), and its conditional covariance is \(\begin{pmatrix}1&0.3\\0.3&4\end{pmatrix}\). A Cholesky factor is \(L=\begin{pmatrix}1&0\\0.3&\sqrt{4-0.3^2}\end{pmatrix}\approx\begin{pmatrix}1&0\\0.3&1.9774\end{pmatrix}\).
With a standard-normal vector \(z=(-1,0.5)^\mathsf{T}\), the deviation is \(Lz=(-1,\ 0.3(-1)+1.9774(0.5))^\mathsf{T}\approx(-1,0.6887)^\mathsf{T}\). The jointly imputed block is therefore \((4,1)^\mathsf{T}+(-1,0.6887)^\mathsf{T}=(3,1.6887)^\mathsf{T}\). The covariance of \(0.3\) enters the second deviation through the first normal draw. Independent marginal sampling would discard this modeled dependence and would not generate draws from the stated joint distribution.
Diagnostics, Alternatives, and Limitations
A joint model is useful when its assumptions provide a reasonable description of the observed data and the conditional distributions needed for missing blocks. Warning signs include implausible imputed ranges, systematic differences between observed and imputed values within comparable availability patterns, unstable covariance estimates, and highly different results under modest changes to the model. For parameter-drawing algorithms, poor convergence or highly autocorrelated draws also undermine the intended uncertainty representation. Diagnostics can reveal a mismatch, but agreement between observed and imputed summaries cannot prove MAR or validate the unobserved values.
High-dimensional omics blocks create a particular limitation: an unrestricted covariance matrix has many parameters, and may be unstable or not estimable when the number of subjects is small relative to the number of features. A structured or regularized covariance model may be necessary, but uncertainty from fitting that structure must still be reflected appropriately. A covariance estimate that is not positive definite cannot be used directly in the Gaussian conditional formula. Transformations and scaling should also be checked: a Gaussian model may be unsuitable for highly skewed, discrete, or strongly bounded measurements without an appropriate modeling choice.
| Approach | What it provides | Main limitation for this task |
|---|---|---|
| Joint multiple imputation | Conditional block draws with variation from predictive and parameter uncertainty. | Depends on a sufficiently well-specified, estimable joint model and missingness assumptions. |
| Conditional-mean or single-value completion | A deterministic completed data set, often convenient for prediction or display. | Does not represent the distribution of plausible missing values or uncertainty across imputations. |
| Complete-case analysis | Uses observed values without imputing missing blocks. | Can discard substantial information and can be biased unless its required assumptions hold. |
| Low-rank matrix completion | Estimates a structured completed matrix under a low-rank assumption. | A completed estimate is not automatically a set of proper predictive draws with parameter uncertainty. |
Structural absence is not ordinary missing data: imputing a value for a modality that does not meaningfully exist for a subject changes the target rather than recovering an unobserved measurement. Likewise, MAR-based imputation does not address nonignorable missingness simply by generating many draws. If the mechanism depends on unobserved values, conclusions require additional assumptions or sensitivity analyses.
Common Pitfalls
- Using only a conditional mean. The mistaken reasoning is that a model’s best prediction is therefore a complete imputation. A mean-only fill suppresses residual variation and can distort later uncertainty; draw from the full conditional distribution instead.
- Holding fitted parameters fixed across all imputations. This omits parameter uncertainty. Draw model parameters as part of a proper multiple-imputation procedure, not just subject-level residuals.
- Imputing features independently. This discards the covariance specified by the joint model. Draw the missing vector jointly from its conditional covariance.
- Treating MAR as established because the model fits observed data. MAR concerns dependence on unobserved values and cannot generally be verified from observed data alone. State the assumption and consider sensitivity to plausible departures.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the Gaussian block model, what do the conditional mean and conditional covariance represent?
- Suppose the conditional covariance is \(\begin{pmatrix}4&1\\1&3\end{pmatrix}\). Give a Cholesky factor and explain one reason to draw the two missing features jointly.
- Why is drawing residuals from a conditional distribution while fixing every model parameter not generally enough for multiple imputation?
- Can observed-data diagnostics establish that missingness is MAR? Why or why not?
- Why is a modality that is structurally undefined for a subject different from an unmeasured modality value?
Solutions. (1) The conditional mean is the expected missing block given observed measurements, covariates, and parameters; the conditional covariance describes the remaining joint variation around that mean. (2) One factor is \(L=\begin{pmatrix}2&0\\0.5&\sqrt{2.75}\end{pmatrix}\), since \(LL^\mathsf{T}=\begin{pmatrix}4&1\\1&0.25+2.75\end{pmatrix}\). Joint drawing retains the modeled covariance between the two features. (3) Fixed parameters omit uncertainty about the fitted model, potentially making completed data sets too similar and understating downstream uncertainty. (4) No. MAR concerns whether missingness depends on unobserved values after conditioning on observed information, which cannot generally be checked using observed data alone. (5) For structural absence, there is no meaningful value to recover; imputing one would alter the target rather than fill in an unobserved measurement.