Why Imputation Must Be Fold-Local
A multi-omics prediction pipeline may encounter subjects for whom an entire modality is unavailable, as well as subjects whose modality is available but has missing individual features. Imputation can provide a complete predictor matrix for a model that requires one. But if the imputation rule is estimated using held-out subjects, the evaluation is no longer an assessment of a pipeline trained without those subjects. Even when imputation does not use outcomes, held-out predictor values can influence the fitted transformation and, in turn, predictions.
For an inductive prediction task, the intended operation is: estimate every data-dependent part of the imputation rule using the training fold, then apply that fixed rule to the held-out fold. This is an instance of fold-local preprocessing. The rule and its fallback choices must be specified so they can also be applied to a future subject without using other future subjects.
Representing Two Levels of Missingness
Let \(x_{ij}^{(m)}\) denote feature \(j\) in modality \(m\) for subject \(i\). Define \(A_i^{(m)}=1\) if modality \(m\) is available for subject \(i\), and \(A_i^{(m)}=0\) if the whole modality is absent. Separately, define \(R_{ij}^{(m)}=1\) if that particular feature is observed and \(R_{ij}^{(m)}=0\) otherwise. A feature cannot be observed when its whole modality is absent, so \(R_{ij}^{(m)}=1\) implies \(A_i^{(m)}=1\). When the modality is available, however, some features can still be missing. The two indicators are therefore not interchangeable.
A simple fold-local rule fills each missing feature with that feature's mean among observed training subjects. For feature \(j\) in modality \(m\), calculate
This estimate is defined only when at least one training subject has the feature observed. For a subject \(i\in V_k\), set the imputed value to \(x_{ij}^{(m)}\) when \(R_{ij}^{(m)}=1\), and to \(\hat{\mu}_{jk}^{(m)}\) when \(R_{ij}^{(m)}=0\). If the entire modality is absent for that subject, all its features are imputed by their respective training-fold estimates. If only a feature is missing, only that feature is filled.
Worked Example: A Feature Missing Within an Available Modality
Suppose modality \(m\) has one feature, and the training fold has five subjects. The modality is available for all five, but the feature values are \(2\), \(4\), missing, \(10\), and missing. Thus \(A_i^{(m)}=1\) for every subject, while the feature-level indicators are \(R_{ij}^{(m)}=1,1,0,1,0\).
The correct training estimate uses the feature-level indicators:
A held-out subject with this feature missing receives an imputed value of approximately \(5.333\). Using the modality indicator in the denominator would incorrectly count all five subjects as having observed this feature, even though two feature values are missing. The mean is therefore based on \(R_{ij}^{(m)}\), not \(A_i^{(m)}\).
Why the Estimate Is Fitted on Training Subjects
The training-fold mean has a simple justification when a constant is used to represent observed values. Among all constants \(c\), the one that minimizes the sum of squared deviations over observed training values is their arithmetic mean. Let \(S_{jk}^{(m)}=\{i\in T_k:R_{ij}^{(m)}=1\}\), with size \(q>0\). Then
where \(\bar{x}_{jk}^{(m)}\) is the mean of the observed values in \(S_{jk}^{(m)}\). The identity follows by expanding each square and using \(\sum_{i\in S_{jk}^{(m)}}(x_{ij}^{(m)}-\bar{x}_{jk}^{(m)})=0\). The second term is nonnegative and is zero only when \(c=\bar{x}_{jk}^{(m)}\); hence the observed training mean minimizes the squared deviations.
This calculation justifies the mean as a least-squares constant for the observed training values. It does not prove that the mean is an accurate estimate of the unobserved values for every missingness mechanism. In particular, if missingness depends on the unobserved feature value, the observed values may not represent the missing ones. Nor does the calculation justify estimating the mean from held-out subjects: the target is a pipeline whose fitted parameters depend only on training data.
Worked Example: Keeping Held-Out Values Out of the Mean
Suppose one feature has observed training-fold values \(2\), \(4\), and \(6\). In the held-out fold, one subject has an observed value of \(100\), while another subject is missing the feature. Assume the evaluation is inductive: each held-out subject is processed using a rule fitted on the training fold, not using other held-out subjects.
The fold-local estimate is
The held-out subject with the missing feature is assigned \(4\). If the value \(100\) from the held-out fold were included, the estimate would instead be
That value is not the training-fold estimate; it depends on a held-out subject. Even though \(100\) is a predictor rather than an outcome, allowing it to affect the imputation rule makes evaluation transductive rather than inductive. It can also change the imputed input and downstream prediction. Under the stated inductive target, the correct imputed value is \(4\), not \(28\).
Using Information From Other Modalities
A constant mean is not the only possible imputation rule. If another modality is available, a conditional model can predict a missing feature from that modality. For example, a regression model can be fitted using training subjects with both the target feature and the auxiliary predictor observed, then applied to held-out subjects with the target feature missing. The model coefficients and any decisions about its predictors must be learned within the training fold. The held-out subject's available predictors may be inputs to the already-fitted rule; held-out subjects must not be used to refit that rule.
Worked Example: Predicting a Missing Feature From Another Modality
Suppose a training fold contains three subjects with observed features \(x\) from one modality and \(y\) from another: \((x,y)=(0,1),(1,3),(2,5)\). Fit the simple linear rule \(y=a+bx\) using these training pairs. The means are \(\bar{x}=1\) and \(\bar{y}=3\). The least-squares slope and intercept are
The fitted rule is \(\hat{y}=1+2x\). A held-out subject with \(x=3\) and missing \(y\) receives \(\hat{y}=1+2(3)=7\). This application assumes that \(x\) is available for the held-out subject and that the fitted linear relationship is a reasonable imputation rule for the target setting. The fit uses only training pairs; a held-out subject with observed \(y\) must not be added to those pairs to refit the coefficients.
More flexible conditional models may use several available modalities, but flexibility does not remove the fold-local requirement. The predictor set, model parameters, and any tuning decisions must be fitted using training data only. If the outcome is used to construct imputations for training predictors, the procedure must also be usable without the outcome at prediction time; otherwise it cannot be applied consistently to held-out or future subjects.
Practical Procedure, Diagnostics, and Alternatives
For every subject and modality, record whether the modality is absent. For each feature, record whether that feature is observed. Do not infer feature observation solely from modality availability.
Estimate each feature mean or conditional imputation model from eligible observed training records. Record the number of records used and any fallback decision.
Keep observed values, fill missing values with the training-fitted rule, and do not recalculate imputation parameters using held-out values.
Look for features with few observed training subjects, folds with no observed value for a feature, implausible imputed ranges, and missingness patterns that differ across folds.
If a feature has no observed values in a training fold, its mean cannot be estimated: its denominator is zero. A defensible response must be specified in advance and applied using only training-fold information. Options include dropping that feature for that fold, using a fallback fixed from external knowledge, or using a prespecified model that can predict it from supported features. Silently borrowing held-out observations to solve the problem is not fold-local imputation. The modeling pipeline must also handle the resulting feature set consistently.
Diagnostics should include observed counts by feature and fold, the fraction of subjects with each modality absent, and summaries of observed and imputed values. Large differences in availability across folds may produce unstable estimates. If missingness is associated with subject characteristics or outcomes, a simple mean can obscure useful structure or fail to represent the missing values. A missingness indicator can be added as a predictor when scientifically and operationally appropriate, but it too must be constructed consistently and assessed for stability. Its predictive association does not establish a causal effect of missingness.
| Approach | Potential advantage | Important limitation |
|---|---|---|
| Fold-local mean or median imputation | Simple and applicable to a single missing feature or an absent block. | Can weaken relationships among features and may poorly represent missing values when missingness is informative. |
| Fold-local conditional imputation | Uses available modalities to predict a missing feature and can preserve some cross-modal structure. | Needs adequate training support and a suitable conditional model; its fitted rule can be unstable. |
| Complete-case analysis | Avoids assigning imputed values. | Can discard many subjects and change the analyzed population; it is not automatically unbiased. |
| Models designed for incomplete modalities | Can use available blocks directly rather than filling every missing entry. | Requires a model and evaluation procedure suited to the missing-block patterns and deployment setting. |
These approaches answer different practical needs. Mean imputation is a transparent baseline, not a universal solution. Conditional imputation can use cross-modal information but introduces additional fitted structure. Complete-case analysis avoids imputation but may lose substantial data. A model that natively accepts incomplete modalities avoids a separate fill-in step, but still needs evaluation that respects the intended availability patterns. In every case, the evaluation procedure should match how predictions will be made for new subjects.
Common Pitfalls and Limits
- Using the modality indicator for a feature mean. A modality can be present while a particular feature is missing. Calculate feature means with \(R_{ij}^{(m)}\), not \(A_i^{(m)}\).
- Fitting imputation once before splitting. Even an outcome-free mean can use held-out predictor information. Refit the rule separately within each training fold.
- Assuming the observed mean represents every missing value. The least-squares argument concerns observed training values; it does not establish that missing and observed values have the same distribution.
- Ignoring a zero denominator. If no training subject has a feature observed, the fold-specific mean is undefined. Use a prespecified training-only fallback or omit the feature according to a documented rule.
- Treating missingness indicators as harmless. They may encode selection or availability patterns that do not persist in the deployment population. Their usefulness and stability require assessment.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A modality is available for four training subjects, but a feature is observed only for values \(3\), missing, \(9\), and missing. What is the fold-local mean imputation value, and which indicator belongs in its denominator?
- Training values for a feature are \(1\), \(5\), and \(6\). A held-out subject has a value of \(30\), while another held-out subject is missing the feature. Under inductive evaluation, what value imputes the missing feature, and why?
- A feature has no observed values in one training fold. Is its training-fold mean zero? State a valid response.
- A conditional imputation model is fitted from training subjects with two modalities observed. Which data may be used to impute a held-out subject whose target modality is missing?
Solutions. (1) Use \(R_{ij}^{(m)}\), since the modality is available for all four but the feature is observed for only two. The estimate is \((3+9)/2=6\). (2) The training mean is \((1+5+6)/3=4\), so impute \(4\). Including the held-out value \(30\) would make the imputation estimate depend on held-out information and would change the inductive procedure. (3) No. Its denominator is zero, so the mean is undefined. Use a prespecified fallback based only on training information, or omit the feature for that fold under a consistent rule. (4) Use the held-out subject's available predictors as inputs to the already-fitted conditional model. Do not use held-out subjects to refit that model or its parameters.