Relative Increase In Variance From Nonresponse

Why Compare the Added Variance With Within-Imputation Variance?

Multiple imputation produces several completed data sets, and the analysis of each one gives an estimate and an estimate of its variance. Those variances describe uncertainty within the completed data sets. The estimates can also differ across imputations, reflecting uncertainty associated with the unobserved values. A useful question is not only how much variance is added, but how large that addition is relative to the within-imputation uncertainty.

The relative increase in variance, \(r\), answers this comparison for a scalar estimate. It is a dimensionless ratio: \(r=0.20\), for example, means that the estimated added variance is 20% of the average within-imputation variance. It does not mean that 20% of observations are missing. Its value depends on the estimate, its within-imputation precision, the between-imputation variation, and the number of imputations.

Definition and Conditions for Use

Suppose the same scalar quantity is estimated in each of \(M\) imputed data sets. Let \(Q_m\) be its estimate in imputation \(m\), and let \(U_m\) be the estimated variance of \(Q_m\) from the analysis of that completed data set. The average within-imputation variance is \(\bar{U}\), and the between-imputation variance is \(B\).

The formula applies to a scalar estimate when all \(M\) analyses target the same estimand, use compatible variance estimates, and use imputations that adequately represent uncertainty about missing values. Rubin’s total variance is \(T=\bar{U}+(1+1/M)B\); this tutorial focuses on expressing its added component relative to \(\bar{U}\). In practice, \(U_m\) should be a valid variance estimate for \(Q_m\), and the imputation procedure should be proper for the inferential task. If the imputation model omits important uncertainty or the imputations have not adequately explored their distributions, \(B\) may not capture the uncertainty it is intended to reflect. The ratio cannot correct those problems.

The denominator must be positive for \(r\) to be defined. The numerator and denominator are both variances, so they have the same squared units and \(r\) has no units. The estimates themselves must be on the same analysis scale across imputations; a variance in one scale cannot sensibly be compared with a variance in another.

Why the Finite-Imputation Correction Appears

The between-imputation variance \(B\) is calculated from a finite set of \(M\) estimates. In Rubin’s total-variance formula, the contribution associated with between-imputation variation is therefore \((1+1/M)B\), rather than just \(B\). The factor approaches 1 as \(M\) grows, but is larger than 1 for any finite \(M\). The relative increase is the ratio of that corrected contribution to the average within-imputation variance.

Thus \(r=0.25\) means the added variance is one quarter of \(\bar{U}\), and the total variance is \(1.25\bar{U}\). This is a statement about variances, not standard errors: the standard error is the square root of a variance, so its relative change is not \(r\).

Interpreting the Size of the Ratio

A larger \(r\) indicates that between-imputation uncertainty makes a larger contribution relative to the within-imputation variance. A small \(r\) indicates that this contribution is comparatively small. Neither size, on its own, diagnoses whether the imputation model is appropriate. For example, a large ratio can be a plausible result when missing values leave substantial uncertainty; a small ratio can arise because missingness contributes little uncertainty, but it can also reflect a poorly specified model or inadequate exploration of the imputation distributions.

The same value of \(B\) can imply different relative increases for different analyses. If \(\bar{U}\) is small, a given added variance is large relative to it; if \(\bar{U}\) is large, that same addition is a smaller proportion. The ratio is therefore specific to the scalar estimate and its analysis, not a general summary of the data set’s missingness.

A Ratio Can Be Read Alongside the Variance Components

Reporting \(r\) without \(\bar{U}\), \(B\), and \(M\) can obscure what drives its value. In particular, identical ratios can result from very different absolute variances. It is often useful to retain the component values in calculations and reports, especially when comparing analyses of the same estimand on the same scale. The following example shows how the ratio and total variance can be calculated together.

Diagnostics, Alternatives, and Limitations

Check the components before treating \(r\) as informative. Confirm that the same estimand and scale were used in every completed-data analysis, that all \(U_m\) values are nonnegative and plausible, and that \(\bar{U}>0\). Recheck the calculation if \(B\) is unexpectedly large, the estimates differ sharply across imputations, or a single estimate dominates the squared deviations. If \(B\) is close to zero, verify that the imputations represent uncertainty and that the computation used distinct, adequately explored imputations. A zero or near-zero estimate of \(B\) is not proof that missingness creates no uncertainty.

The formula includes a finite-\(M\) adjustment, but it does not remove Monte Carlo error from estimating \(B\). With few imputations, \(B\) can be unstable, and hence \(r\) can also be unstable. Increasing \(M\) can improve the numerical stability of pooled calculations; it cannot repair an unsuitable imputation model, inconsistent analysis models, or imputations that fail to capture relevant uncertainty. Likewise, if successive saved imputations are highly dependent, the nominal number \(M\) may overstate the amount of independent information in the set. These are reasons to assess the imputation and computation, not to reinterpret the formula.

A common arithmetic alternative is the uncorrected ratio \(B/\bar{U}\). It is smaller than \(r\) by the factor \(1+1/M\), and is not the finite-imputation-adjusted quantity defined here. Reporting \(B\) and \(\bar{U}\) separately can help explain the result, but neither alone gives the relative increase. A complete-case analysis avoids combining estimates across imputations but targets the same population quantity only under additional conditions; a single-imputation analysis does not represent between-imputation uncertainty. Those approaches are not substitutes for correctly calculating \(r\) from a multiple-imputation analysis.

Finally, \(r\) is not the proportion of records with missing data, nor should it be read as a percentage increase in the standard error. If \(r=0.48\), the variance is multiplied by \(1.48\) relative to \(\bar{U}\); the corresponding standard error is multiplied by \(\sqrt{1.48}\), not by \(1.48\). The ratio is a component comparison, not a complete assessment of model quality or a standalone inferential conclusion.