Training-Fold Imputation of Missing Predictors

Why Imputation Must Be Fold-Local

Genomic predictor matrices often contain missing entries: an assay may fail for a sample at some genes, or a gene may be unavailable for a subset of samples. A prediction method generally requires a defined input, so missing predictors need a rule for filling them. In cross-validation, that rule is part of the fitted procedure. If the rule is estimated using held-out samples, the evaluation no longer represents a procedure trained only on its training data.

For a gene-wise mean rule, the relevant estimate is the mean among observed training-fold values for that gene. For a learned imputation model, it is the model fitted using eligible training-fold data. In either case, the fitted rule is then applied to missing entries in both the training and validation data. The validation samples may supply their own observed predictor values as inputs to that fixed rule; they must not be used to estimate its parameters.

This distinction is about where the rule is learned, not whether a validation sample’s observed predictors can be used. For example, a model that predicts a missing gene from other measured genes can take a validation sample’s observed genes as inputs. It cannot be refitted using the collection of validation samples, even if those samples have no outcomes attached.

Gene-Wise Means: Fit, Then Apply

Let \(X_{ij}\) denote the value for sample \(i\) and gene \(j\), when observed. Let \(T\) denote the training indices in a fold and \(V\) its validation indices. Define \(O_{Tj}\) as the training samples for which gene \(j\) is observed. If \(O_{Tj}\) is nonempty, the training-fold mean for gene \(j\) is:

Every missing value for gene \(j\) in the training data is replaced by \(\hat{\mu}_{Tj}\), and every missing value for gene \(j\) in validation is replaced by that same \(\hat{\mu}_{Tj}\). Observed values are left unchanged. The validation values are not included in the sum, even when they are observed.

There is no training-fold mean if gene \(j\) is missing for every training sample: then \(|O_{Tj}|=0\), and the formula is undefined. The analysis must specify a fallback that does not learn from validation data, such as removing that gene for this fold under a training-only rule, or using a prespecified constant on the appropriate scale. Silently substituting a validation mean does not solve the problem; it introduces held-out information.

A Learned Imputation Model

Instead of a separate mean for each gene, one may fit a model that predicts a gene from other predictors. For example, to impute gene \(B\) from gene \(A\), a simple model might be fitted to training samples where both are observed, then used to predict \(B\) for samples where \(B\) is missing and \(A\) is observed. The model form, variables, and estimated coefficients must be determined without validation samples. Any tuning or selection involved in the imputation model must also use training data only.

A conditional model can preserve some relationships between genes that mean imputation removes, but its assumptions matter. A linear conditional mean model assumes the expected value of the target gene given the input genes is adequately represented by the specified linear function over the relevant range. Its training pairs must be sufficiently informative to estimate that function. If the validation samples have input patterns far outside the training range, predictions may be extrapolations and unreliable.

Some imputation methods use several genes jointly, or estimate parameters iteratively. The same boundary applies: fit all quantities from the training fold and then apply the fitted procedure to validation cases. A method that requires joint estimation across the validation samples is not automatically fold-local merely because it does not use their outcomes.

Why Fold-Specific Missingness Matters

Missingness can vary across folds. For example, batches may differ in assay quality, or a small sample may happen to contain many failures for a particular gene. Consequently, the observed training mean, the number of observed training values, and the proportion of missing validation entries can all differ from fold to fold. These differences are part of the data situation encountered by each fitted model, not a reason to pool folds when estimating imputation rules.

Mean imputation does not by itself require that missingness be completely at random for the arithmetic to be performed. But whether the imputed values represent plausible predictors, and whether validation missingness resembles future use, depends on the missingness process. If missingness is associated with the unobserved gene value, for example, the observed training mean may systematically misrepresent the missing values. Cross-validation cannot make that imputation bias disappear.

Justification: Why Validation-Fitted Rules Leak

The reason for the training-only boundary can be stated directly. For a given fold, the intended fitted rule is a function of the training data, say \(g_T\). Validation imputation is then \(g_T(X_V)\), where \(X_V\) supplies inputs to the rule but does not determine its fitted parameters. A rule estimated from both training and validation data is instead a function \(g_{T,V}\). It is not the rule that could have been learned from the training data alone.

If the second mean differs from the first, an imputed validation value depends on validation measurements through the estimated mean. This is information flow from the held-out fold into model fitting. It may make evaluation optimistic, though the direction and size of the effect are not guaranteed: leakage can alter predictions in ways that help or harm a particular score.

Diagnostics, Alternatives, and Limitations

Useful checks include, for each fold and gene, the count of observed training values, the training and validation missingness rates, and whether a mean or model could be fitted. For learned imputers, also check convergence, coefficient stability where relevant, and whether validation inputs lie in a range supported by training data. A fold with no observed training values for a gene, or with too few complete pairs for a conditional model, needs an explicit training-only fallback or a documented failure; it should not be repaired by consulting validation values.

A common mistake is to impute the entire matrix once, then split it into folds, on the grounds that imputation is “unsupervised” because it does not use outcomes. This is still leakage: held-out predictor values have influenced the rule. The correction is to estimate the imputation rule anew within each training fold. Conversely, applying a fixed training-fitted conditional model to a validation sample’s observed predictors is not leakage; those predictors are the inputs available when making that sample’s prediction.

The choice among imputation methods is a modeling decision, not merely a computational detail. Mean imputation is simple and transparent, but fills missing values with one constant per gene and can reduce variability. A conditional imputer can retain more structure when its model is adequate, but demands enough training information and compatible validation inputs. Complete-case analysis avoids filling values but can be inefficient or unrepresentative when missingness is systematic.

In predictive evaluation, the imputation strategy should match what will be possible at deployment. If an outcome is unavailable for a new sample, it cannot be used as an input to impute that sample’s predictors. Outcome-assisted imputation in training is not directly applicable to held-out prediction unless the same outcome information is genuinely available at prediction time. Likewise, a validation sample’s predictors may be used only to the extent those predictors would be available under the intended use.

The cross-validation estimate therefore evaluates a full procedure that includes how missing predictors are handled. If missingness differs across folds, report or inspect that variation rather than treating every fold’s imputation rule as identical. A score difference may arise from different training samples, different missingness burdens, or both; fold-local imputation prevents information leakage, but it does not guarantee stable performance or remove the consequences of distribution shift.