Single-Value Imputation for Numeric Predictors

Why a Single Replacement Needs a Rationale

A numeric predictor with missing entries can prevent a modeling procedure from using some rows, or in some implementations from fitting at all. A simple alternative is single-value imputation: choose one value and substitute it for every missing entry in that predictor. The arithmetic is straightforward; the statistical consequences are not. The replacement changes the predictor’s distribution, can affect relationships with an outcome, and may suggest more precision than the data support.

The three common choices are the observed mean, the observed median, and a fixed value chosen for a substantive reason. None is universally best. A defensible choice depends on what the predictor measures, why values are missing, how the model will be used, and whether the replacement is calculated using only the data available at the time of prediction.

Definition: Suppose a numeric predictor has $n$ observed values and $m$ missing values. Single-value imputation replaces each missing entry with the same constant $c$, producing a completed set of $N=n+m$ values. Mean imputation uses the observed mean as $c$; median imputation uses the observed median; fixed-value imputation uses a prespecified value with a context-based justification.

How the Choice Changes the Completed Data

Let $\bar{x}_{\text{obs}}$ be the mean of the $n$ observed values. After replacing the $m$ missing values with $c$, the completed-data mean is the sum of the observed values and replacements divided by $N$:

\bar{x}_{\text{comp}} = \frac{n\bar{x}_{\text{obs}}+mc}{n+m}.

This identity immediately clarifies one property of mean imputation: when $c=\bar{x}_{\text{obs}}$, the completed mean equals the observed mean. That does not mean the completed data reproduce the unknown values or the true population mean. It only means the replacement values balance the observed values around their own mean.

The variance also changes. Let $s_{\text{obs}}^2$ denote the sample variance of the observed values, using denominator $n-1$. For $n>1$, the completed sample variance is

s_{\text{comp}}^2 = \frac{(n-1)s_{\text{obs}}^2+ \frac{nm}{n+m}(\bar{x}_{\text{obs}}-c)^2} {n+m-1}.

This formula describes the sample variance of the filled-in numbers, not an estimate that accounts for uncertainty about the missing values. In particular, it does not restore the variability that would have been present if those values had been observed.

Why Mean Imputation Shrinks the Variance

The variance result follows by separating variation among the observed values from the shift between each group’s mean and the completed mean. Write $\bar{x}_{\text{comp}}$ for the completed mean. The observed values contribute their deviations around $\bar{x}_{\text{obs}}$, plus the shift from $\bar{x}_{\text{obs}}$ to $\bar{x}_{\text{comp}}$. The $m$ replacement values contribute only their shift from $c$ to $\bar{x}_{\text{comp}}$:

\sum_{\text{completed }i}(x_i-\bar{x}_{\text{comp}})^2 = (n-1)s_{\text{obs}}^2 +n(\bar{x}_{\text{obs}}-\bar{x}_{\text{comp}})^2 +m(c-\bar{x}_{\text{comp}})^2.

Substitute $\bar{x}_{\text{comp}}=(n\bar{x}_{\text{obs}}+mc)/(n+m)$. Then $\bar{x}_{\text{obs}}-\bar{x}_{\text{comp}}=\frac{m}{n+m}(\bar{x}_{\text{obs}}-c)$, while $c-\bar{x}_{\text{comp}}=-\frac{n}{n+m}(\bar{x}_{\text{obs}}-c)$. The two between-group terms therefore sum to $\frac{nm}{n+m}(\bar{x}_{\text{obs}}-c)^2$. Dividing the total squared deviation by $n+m-1$ gives the stated sample variance formula.

When mean imputation is used, $c=\bar{x}_{\text{obs}}$, so the between-group term is zero. The observed sum of squared deviations remains, but it is divided by the larger completed-sample denominator $n+m-1$. Thus the completed sample variance is $\frac{(n-1)s_{\text{obs}}^2}{n+m-1}$, smaller than $s_{\text{obs}}^2$ when $m>0$. The repeated mean values also create a pile-up at one point in the empirical distribution.

Median imputation can be less sensitive than mean imputation to an extreme observed value when choosing a typical replacement. But it still puts all missing entries at one point. It does not generally preserve the observed mean, and it does not reconstruct the spread or shape of the missing values. A fixed value can move the completed mean and variance in either direction, depending on how far that value is from the observed mean.

Worked Example: Comparing Mean, Median, and Zero Replacement

A hypothetical pricing dataset has observed values $2,4,4,6,14$ for a numeric predictor and two missing values. Calculate the completed mean and sample variance under mean, median, and zero imputation. The zero replacement is included for comparison, not assumed to be appropriate.

Step 1: Summarize the observed values. There are $n=5$ observed values and $m=2$ missing values, so $N=7$. Their sum is $2+4+4+6+14=30$, giving $\bar{x}_{\text{obs}}=30/5=6$. In sorted order the values are $2,4,4,6,14$, so the median is $4$. The squared deviations from 6 are $16,4,4,0,64$, which sum to $88$. Therefore, $s_{\text{obs}}^2=88/(5-1)=22$.

Step 2: Mean imputation. Fill both missing values with 6. The completed mean is $(30+2(6))/7=42/7=6$. The replacements add no squared deviation from 6, so the completed sum of squared deviations is 88. The completed sample variance is $88/(7-1)=88/6=14.667$, rounded to three decimal places.

Step 3: Median imputation. Fill both missing values with 4. The completed mean is $(30+2(4))/7=38/7=5.429$, rounded to three decimal places. Using the variance formula, the completed sum of squared deviations is $88+\frac{5(2)}{7}(6-4)^2=88+\frac{40}{7}=\frac{656}{7}$. Divide by 6: $s_{\text{comp}}^2=\frac{328}{21}=15.619$, rounded to three decimal places.

Step 4: Zero imputation. Fill both missing values with 0. The completed mean is $(30+2(0))/7=30/7=4.286$, rounded to three decimal places. The completed sum of squared deviations is $88+\frac{10}{7}(6-0)^2=88+\frac{360}{7}=\frac{976}{7}$. Divide by 6: $s_{\text{comp}}^2=\frac{488}{21}=23.238$, rounded to three decimal places.

Check and interpretation. The completed means are 6, 5.429, and 4.286; each equals the sum of its seven completed values divided by 7. The mean replacement has the smallest variance because it adds no deviation from the observed mean and increases the variance denominator. The zero replacement has the largest variance here because zero is far from 6. These calculations describe the completed numbers; they do not establish which choice best represents the missing values. The common mistake is to treat the smallest completed variance as evidence that mean imputation is most accurate.

When Each Replacement Is Defensible

Choice	Potential justification	Important limitation
Observed mean	A simple baseline when a central arithmetic value is suitable and the imputation method is clearly documented.	Sensitive to extreme observed values; preserves the observed mean but compresses variability and creates a point mass.
Observed median	A robust typical-value choice for a skewed predictor or one with extreme values, when a simple baseline is sufficient.	Does not generally preserve the mean or the distribution; every replacement is still identical.
Fixed value	A prespecified value has a direct substantive meaning, such as zero when the data definition guarantees that a blank truly means none.	Can distort summaries and relationships if the fixed value does not match the meaning of the missing entry.

A fixed zero is not defensible merely because zero is easy to interpret or because the predictor cannot be negative. A missing measurement is not automatically a zero measurement. Before choosing a constant, distinguish an unrecorded value from a known absence or a documented data-coding rule. Likewise, mean or median imputation should not be presented as a correction for a missingness process that makes observed values systematically unrepresentative. The earlier material on missingness assumptions and complete-case selection risks motivates checking why values are missing before settling on a simple replacement.

In a predictive workflow, calculate a data-derived replacement from the training data and then use that same value for validation or future records. Calculating it from all available rows lets information from validation data influence the training procedure. A fixed value specified in advance by the predictor’s meaning does not need to be estimated from the data, but it should still be applied consistently.

Worked Example: Avoiding Validation-Set Leakage in R

A hypothetical model uses a numeric predictor. Its training values are $10,12,14,\text{NA}$, and its validation values are $\text{NA},20$. Impute missing entries with the training observed mean. Show the correct calculation and the result of incorrectly estimating the mean from both sets.

Correct method. In the training set, the observed values sum to $10+12+14=36$, and there are 3 observed values. The training mean is $36/3=12$. Replace the training NA with 12, giving training values $10,12,14,12$. Apply that same fitted value to the validation set, giving $(12,20)$. The observed validation value 20 is left unchanged.

In R, the training value can be calculated as mean(train$x, na.rm = TRUE). Save the result as, for example, fill_x, then use train$x[is.na(train$x)] <- fill_x and validation$x[is.na(validation$x)] <- fill_x. The missing-value test is applied to each set, but the replacement value is estimated only from training observations.

Incorrect method and correction. If the observed validation value 20 is included in the calculation, the combined observed mean is $(10+12+14+20)/4=14$. Using 14 to fill the validation NA allows a validation-set value to affect preprocessing. The correct validation fill remains 12 because the model’s preprocessing rule was fitted on the training set. This example’s common trap is to believe that a more data-rich estimate is always preferable; for validation, using held-out information changes what the evaluation measures.

Practical Checks Before Using Single-Value Imputation

Single-value imputation is easy to implement, but it should be treated as a modeling decision rather than a harmless data-cleaning edit. Before proceeding, check the following:

Confirm the meaning of missingness. A value omitted because of a data-collection failure is not necessarily equivalent to zero or to a typical observed value.
Inspect the observed distribution. Skewness or extreme observations can make the mean a poor typical-value choice; the median may be more stable as a replacement but does not restore distributional shape.
Track how many values are replaced. A replacement used for a very small fraction of records may have different consequences from one used for a large fraction. In either case, the same-value pile-up remains.
Fit preprocessing on training data only. Save the chosen training replacement and reuse it for validation and later prediction.
Describe the limitation. The completed predictor contains invented repeated values, and ordinary summaries of it do not express uncertainty about what the missing values might have been.

Mean, median, and fixed-value imputation can be useful as transparent baselines or when a domain rule supports the choice. They are not interchangeable, and no one choice is justified by convenience alone. In particular, mean imputation’s preservation of the observed mean is an arithmetic property, not evidence that missing values are representative or that the predictor-outcome relationship is preserved.

Key takeaway: Choose a replacement whose meaning is defensible, calculate data-derived values from training observations only, and interpret the completed distribution cautiously. Single-value imputation creates repeated values and generally understates the variability that uncertainty about missing entries would contribute.

Check Your Understanding

For each problem, calculate using the stated data and distinguish an arithmetic result from a justification for the imputation choice.

A numeric predictor has observed values $3,5,7$ and one missing value. Find its mean replacement and the completed-data mean. (Answer: Mean replacement $=5$; completed mean $=5$.)
Observed values are $1,2,2,9$, with one missing value. Find the observed median and the completed mean after median imputation. (Answer: Median $=2$; completed mean $=16/5=3.2$.)
A predictor has observed values $2,4,6$ and one missing value. Calculate the completed sample variance after mean imputation. Use the sample-variance denominator $N-1$. (Answer: Observed mean $=4$, observed sample variance $=4$, completed sum of squared deviations $=8$, and completed sample variance $=8/3\approx2.667$.)
A training set has observed values $8,10$ and one missing value. A validation set has an observed value of 40 and one missing value. If the rule is training-mean imputation, what value fills each missing entry, and what incorrect value would result from pooling observed training and validation values first? (Answer: Correct fill $=9$ in both sets; pooled observed mean $=(8+10+40)/3=58/3\approx19.333$, which must not be used for this validation procedure.)
A system’s documented data definition says a blank predictor entry means the policy has no optional rider, and the predictor counts optional riders. Is zero replacement defensible under that rule, and what important check should still be made? (Answer: Yes, if the documented rule truly guarantees that blank means none; verify that the rule applies to these records and that blanks are not also used for unrecorded or failed measurements.)

What You'll Learn