Within-Imputation Variance And Its Pool

Why Pool Complete-Data Variances?

Multiple imputation produces $M$ completed data sets. The same analysis is then applied to each one, giving $M$ estimates of the target quantity and a complete-data variance estimate for each estimate. This tutorial focuses on pooling those variance estimates. The pooled within-imputation variance summarizes the sampling uncertainty each completed-data analysis reports, averaged across the imputations.

This quantity is important, but it is not the full uncertainty summary for a multiply imputed analysis. It describes uncertainty within the completed-data analyses; it does not describe how much their point estimates differ from one imputation to another. That distinction is essential when reporting results. The separate contribution from differences among estimates is addressed in the following material on between-imputation variance.

Definition: Let $Q_i$ be the estimate of the target quantity from completed data set $i$, and let $U_i$ be its estimated complete-data variance, for $i=1,\ldots,M$. The pooled within-imputation variance is the arithmetic mean of the $M$ variance estimates: $$ \bar{U}=\frac{1}{M}\sum_{i=1}^{M}U_i. $$ Each $U_i$ is a variance estimate, not a standard error. If the analysis reports a standard error $SE_i$ for $Q_i$, then $U_i=SE_i^2$.

The calculation has a straightforward interpretation: treat each completed-data analysis as providing an estimate of sampling variance for the same target, then average those estimates with equal weight. A variance has squared units. For example, if $Q_i$ is measured in millimetres, $U_i$ and $\bar U$ are measured in square millimetres. The quantity $\sqrt{\bar U}$ is on the original scale, but it summarizes only the average within-imputation variance; it should not, by itself, be presented as the full standard error for the multiply imputed estimate.

Conditions for a Meaningful Average

The arithmetic is simple, but interpretation depends on the analyses being comparable. Use the same estimand, outcome and predictor definitions, model specification, and variance-estimation method in every completed data set. Each $U_i$ must estimate the variance of the corresponding $Q_i$ under that same complete-data analysis. If, for instance, one data set uses a different outcome coding or a different adjustment set, the resulting variances do not automatically estimate a common quantity and should not simply be averaged.

The complete-data variance estimates must also be meaningful under the model used for each completed-data analysis. This includes the assumptions required by that analysis—for example, the specified model and its variance estimator must be appropriate for the data and estimand. If a model-based standard error relies on a particular variance assumption, that assumption is relevant to $U_i$ as well. The within-imputation averaging formula does not repair a misspecified analysis model or a variance estimator that is inappropriate for the design.

For the within-imputation variance to have its intended role in multiple-imputation inference, the imputation and analysis procedures must also support inference for the same target. The $M$ completed data sets should be generated by the intended imputation procedure, and the analysis should be applied consistently to each. The averaging identity itself is just arithmetic; its statistical interpretation relies on these conditions. It does not certify that an imputation model is adequate.

Worked Example: Averaging Standard Errors Correctly

Suppose a hypothetical analysis estimates a mean difference in symptom score, measured in points, from $M=4$ imputed data sets. The complete-data standard errors are $0.18$, $0.22$, $0.20$, and $0.16$ points. Assume that the same mean-difference estimand and analysis were used in all four data sets, and that each standard error is appropriate for its complete-data analysis.

First square each standard error to obtain its variance estimate:

\begin{aligned} U_1&=0.18^2=0.0324,\\ U_2&=0.22^2=0.0484,\\ U_3&=0.20^2=0.0400,\\ U_4&=0.16^2=0.0256. \end{aligned}

Then average the four variances:

\bar{U} =\frac{0.0324+0.0484+0.0400+0.0256}{4} =\frac{0.1464}{4} =0.0366\ \text{points}^2.

The pooled within-imputation variance is $0.0366$ square points. Its square root is $\sqrt{0.0366}=0.1913$ points, a within-only scale summary. It is not obtained by treating the variance itself as a standard error, and it is not a complete multiple-imputation standard error.

Why the Order of Operations Matters

Averaging standard errors is not the same operation as averaging variances and then taking a square root. For a given set of positive standard errors, the pooled within-imputation variance is defined on the variance scale. The correct sequence is to square each standard error, average the squared values, and, if useful, take the square root of that average afterward.

A short justification follows from the definition of the complete-data variance estimate: $U_i=SE_i^2$. Substituting this expression into the definition of $\bar U$ gives

\bar{U}=\frac{1}{M}\sum_{i=1}^{M}SE_i^2, \qquad \sqrt{\bar{U}}= \sqrt{\frac{1}{M}\sum_{i=1}^{M}SE_i^2}.

By contrast, the arithmetic mean of the standard errors is $\frac{1}{M}\sum_i SE_i$. In general, the square root of the mean of squares is not the mean:

\sqrt{\frac{1}{M}\sum_{i=1}^{M}SE_i^2} \ne \frac{1}{M}\sum_{i=1}^{M}SE_i.

They coincide when all the standard errors are equal, but not generally. This is a calculation issue, not a choice between two equivalent pooling conventions.

Worked Example: How Averaging Standard Errors Changes the Result

Consider four hypothetical complete-data standard errors for one coefficient, all measured in the same units: $0.10$, $0.10$, $0.30$, and $0.30$. Assume the same coefficient and analysis are used in every imputation. The variance estimates are $0.10^2=0.0100$, $0.10^2=0.0100$, $0.30^2=0.0900$, and $0.30^2=0.0900$, in squared coefficient units.

The correct pooled within-imputation variance is

\bar{U}=\frac{0.0100+0.0100+0.0900+0.0900}{4} =\frac{0.2000}{4} =0.0500.

Taking its square root gives $\sqrt{0.0500}=0.2236$ in the coefficient's units. If instead the four standard errors were averaged directly, the result would be

\frac{0.10+0.10+0.30+0.30}{4}=0.2000.

The two summaries differ: $0.2236$ is the square root of the average variance, whereas $0.2000$ is the average standard error. Only the first follows from the definition of $\bar U$. In this example, the difference is noticeable because the standard errors vary appreciably across imputations.

What Is—and Is Not—Being Averaged

The quantities $U_i$ describe estimated sampling variance within the separate complete-data analyses. They are not the point estimates $Q_i$, and averaging $U_i$ does not measure the spread of those point estimates. Two distinct patterns can therefore occur: complete-data analyses may have similar variances but noticeably different estimates, or their estimates may be similar while their variance estimates differ. The pooled within-imputation variance summarizes only the first kind of quantity: the $U_i$'s.

This separation also clarifies the comparison with a single completed-data analysis. Using one completed data set supplies one $U_i$, not an average over the imputations. Repeating the analysis and averaging $U_i$ summarizes within-imputation variance across the completed data sets, but it still does not capture every source of uncertainty in multiple imputation. Do not treat $\bar U$ alone as a full pooled variance, a final confidence interval, or a test statistic.

Likewise, this calculation is not a reason to average results from analyses that target different parameters. If a coefficient is absent in some fits, a categorical variable has different reference levels, or models use different covariates, first determine whether the estimates and variance estimates are genuinely on the same target and scale. An arithmetic mean cannot make incomparable quantities comparable.

Worked Example: Pooling Variances for a Regression Coefficient

Suppose a hypothetical logistic regression is fit in $M=5$ completed data sets. In each fit, the target is the same treatment coefficient on the log-odds scale, and the model uses the same specification and variance estimator. The reported standard errors are $0.12$, $0.15$, $0.10$, $0.13$, and $0.14$ log-odds units. For this example, assume the model-based variance estimates are suitable for the analysis.

Square each standard error:

\begin{aligned} U_1&=0.12^2=0.0144, & U_2&=0.15^2=0.0225,\\ U_3&=0.10^2=0.0100, & U_4&=0.13^2=0.0169,\\ U_5&=0.14^2=0.0196. \end{aligned}

The sum is $0.0144+0.0225+0.0100+0.0169+0.0196=0.0834$. Thus,

\bar{U}=\frac{0.0834}{5}=0.01668\ \text{(log-odds units)}^2.

The square root, $\sqrt{0.01668}=0.1292$ log-odds units, summarizes the square root of the average complete-data variance. It remains a within-imputation summary only. It does not describe how much the estimated treatment coefficients varied across the five completed data sets.

Diagnostics and Common Mistakes

Before interpreting a pooled within-imputation variance, check the inputs rather than relying on the final average alone. The following checks help identify calculation errors and incompatible analyses:

Check the scale and target. Confirm that every $Q_i$ estimates the same quantity, in the same units and parameterization.
Check whether the input is a variance or a standard error. If the software reports $SE_i$, square it before averaging. If it reports a variance-covariance matrix, use the relevant variance entry, not its square root.
Check all $M$ contributions. There should be one appropriate $U_i$ for each completed-data analysis included in the pool. An accidental omission or repeated value changes the average.
Review unusual values. A negative variance is invalid. An unexpectedly large or small positive value may signal convergence problems, separation, a coding discrepancy, or an unsuitable model variance estimator; investigate rather than automatically deleting it.
Use a consistent analysis. Differences in model specification, coding, or variance-estimation method can make the inputs incomparable even when they share a label.

A frequent mistake is to calculate $U_i$ correctly, then call $\bar U$ the total uncertainty of the multiply imputed estimate. The correction is to label it precisely: $\bar U$ is the average complete-data variance estimate, or pooled within-imputation variance. It is one component of multiple-imputation uncertainty, not a standalone final uncertainty measure.

Key takeaway: For each completed data set, use the variance estimate for the same target and analysis. Average those variances with equal weight: $\bar U=M^{-1}\sum_i U_i$. If only standard errors are available, square them first. Keep this within-imputation quantity distinct from both the spread of point estimates and the full uncertainty summary.

Check Your Understanding

Try each question, then compare your work with the solutions below.

Three completed-data analyses report standard errors of $0.20$, $0.30$, and $0.40$ units. Calculate $\bar U$, including its units.
Why is averaging those three standard errors not the definition of pooled within-imputation variance?
Can $\bar U$ be interpreted as the variation among the $Q_i$ estimates? Explain.
One of five completed-data analyses uses a different covariate adjustment set from the other four. What should be checked before averaging the five variance estimates?

Solutions. (1) The variance estimates are $0.20^2=0.0400$, $0.30^2=0.0900$, and $0.40^2=0.1600$ square units. Therefore, $\bar U=(0.0400+0.0900+0.1600)/3=0.09667$ square units. (2) Each reported standard error must first be squared to give its variance estimate; the definition averages those variances, not the standard errors. (3) No. $\bar U$ averages the estimated sampling variances within completed-data analyses. It does not measure how far the point estimates $Q_i$ are spread across imputations. (4) Check whether all five analyses still target the same estimand and whether their variance estimates are comparable. If the different adjustment set changes the target or analysis in a way that makes the variances incomparable, do not blindly average them; resolve the specification issue first.