Why a Pooled Interval Needs Careful Reading
Multiple imputation does not produce just one completed data set. It produces several plausible completed versions of the data, each of which can give a slightly different regression estimate. A pooled confidence interval has to reflect both the uncertainty within each completed data set and the variation in estimates across them. Reading only the pooled coefficient, or treating the imputations as if they were independent new samples, misses this distinction.
The width of the interval has a practical interpretation: it reflects uncertainty in the coefficient under the analysis and imputation procedures used. Some of that uncertainty would remain even if the missing values were known; some arises because they are not known. A larger number of imputations can reduce the extra uncertainty caused by using only a finite number of imputations, but it cannot by itself make the imputation assumptions more credible.
What the Pooled Quantities Represent
Suppose there are \(m\) imputed data sets. Fit the same regression model to each one. Let \(q_j\) be the estimate of a particular regression coefficient from imputed data set \(j\), and let \(U_j\) be its estimated variance. The standard error in that data set is \(\sqrt{U_j}\). The pooled estimate is the average of the \(m\) estimates.
The within-imputation term \(\bar{U}\) averages the regression variance estimates from the completed data sets. The between-imputation term \(B\) measures how much the coefficient estimates vary across those data sets. The multiplier \(1+1/m\) includes a finite-imputation adjustment: estimates from a finite set of imputations introduce additional Monte Carlo uncertainty.
A conventional approximate confidence interval uses a \(t\) critical value with degrees of freedom estimated from the same quantities. In the large-sample version of Rubin’s degrees-of-freedom approximation,
Here \(\nu\) is the approximate degrees of freedom, and \(t_{1-\alpha/2,\nu}\) is the corresponding \(t\) quantile. When \(\nu\) is large, the quantile is close to the standard normal quantile. The approximation is most straightforward when the regression estimate is approximately normal and the within-data variance estimates are appropriate. Software may use additional small-sample adjustments in settings where they are needed; the interval and degrees of freedom should be interpreted according to the method actually used.
Worked Examples: Calculating and Interpreting Pooled Intervals
Worked Example: Pooling Five Regression Estimates
In a hypothetical analysis, a regression coefficient measures the change in a health score, in points, for a one-unit increase in a predictor. Five imputed data sets give coefficient estimates \(1.8,\ 2.0,\ 1.9,\ 2.2,\ 2.1\). Their estimated variances are \(0.25,\ 0.24,\ 0.26,\ 0.25,\ 0.24\), respectively. Calculate an approximate 95% pooled confidence interval.
The pooled estimate is
The average within-imputation variance is
The squared deviations from \(2.0\) are \(0.04,\ 0,\ 0.01,\ 0.04,\ 0.01\). Thus
With \(m=5\), the pooled variance and standard error are
The degrees-of-freedom approximation is
The 95% \(t\) critical value is approximately \(1.967\), so the margin of error is \(1.967(0.5273)\approx1.037\). The interval is \(2.0\pm1.037\), or approximately \((0.963,\ 3.037)\) points per predictor unit. Under the imputation and regression assumptions, this procedure gives a 95% confidence interval for the coefficient. It does not mean there is a 95% probability that the fixed coefficient lies in this particular calculated interval.
How Missing Information Adds Uncertainty
The term \((1+1/m)B\) is the part of the pooled variance added because the coefficient differs across imputations, including the finite-\(m\) adjustment. If imputations produce nearly identical coefficients, \(B\) is small and this contribution is small. If they produce more variable coefficients, the contribution is larger. The interval therefore responds not only to the average regression precision within completed data sets, but also to how much the missing values leave the estimate unsettled.
One useful descriptive quantity is the fraction of the pooled variance contributed by the between-imputation term, \((1+1/m)B/T\). This is a variance share, not a probability that the imputation model is correct. A substantial share indicates that variation across the imputations contributes meaningfully to the reported uncertainty.
Worked Example: Seeing the Added Uncertainty
Suppose a hypothetical coefficient has \(\bar{q}=0.60\), \(\bar{U}=0.16\), and \(B=0.09\) from five imputations. Compare the pooled uncertainty with the within-imputation uncertainty alone.
The within-imputation standard error alone is \(\sqrt{0.16}=0.400\). But the pooled variance is
The pooled standard error is larger than \(0.400\). The between-imputation contribution is \(0.108\), which is \(0.108/0.268\approx0.403\), or about 40.3% of the pooled variance. Thus variation between the imputations is an important part of the uncertainty in this example.
For an approximate 95% interval, \(r=(1+1/m)B/\bar{U}=0.108/0.16=0.675\), giving \(\nu=4(1+1/0.675)^2\approx24.6\). Using a critical value of about \(2.060\), the interval is \(0.60\pm2.060(0.5177)\), or approximately \((-0.466,\ 1.666)\). The interval describes uncertainty in the coefficient under the fitted regression and imputation procedure; its width reflects variation both within and between completed data sets.
More Imputations: What Changes and What Does Not
For a fixed set of underlying imputation results, increasing \(m\) reduces the finite-imputation adjustment \(B/m\), since \(T=\bar{U}+B+B/m\). It can also stabilize estimates of the between-imputation variability. But adding imputations does not make the term \(B\) vanish: as \(m\) grows, the pooled variance approaches \(\bar{U}+B\), not \(\bar{U}\). The spread in coefficient estimates reflects uncertainty associated with missing values, not merely a shortage of imputations.
Worked Example: Increasing the Number of Imputations
Consider a hypothetical coefficient with \(\bar{q}=0.60\), \(\bar{U}=0.16\), and \(B=0.09\). Compare the pooled standard error using five imputations, fifty imputations, and the limiting variance as the number of imputations becomes very large. Treat \(\bar{U}\) and \(B\) as approximately stable for this illustration.
With five imputations,
With fifty imputations,
As \(m\) tends to infinity, \(B/m\) tends to zero, but \(B\) remains:
The reduction from five to fifty imputations decreases the standard error in this illustration, but it does not remove the between-imputation variance. Even infinitely many imputations would leave a standard error of \(0.500\), given these values of \(\bar{U}\) and \(B\). More imputations reduce finite-imputation error; they do not reveal the missing values or erase uncertainty about them.
Why a Large Number of Imputations Is Not a Cure
A pooled interval is conditional on the analysis choices and assumptions that produced it. In particular, its interpretation relies on the imputation procedure representing the uncertainty in missing values adequately, and on the regression analysis being suitable for the question. In a standard missing-at-random framework, the missingness mechanism must be ignorable given the observed information under the relevant conditions; the imputation model must also preserve the relationships needed for the regression analysis. These assumptions are not verified simply by producing many imputations.
For example, suppose an important predictor is omitted from the imputation model, or an interaction needed by the regression is not represented. Repeating that inadequate imputation model many times can give a stable pooled estimate and a precise-looking interval. Stability across imputations is not evidence that the model adequately represents the data. If missingness depends on unobserved values in a way not accounted for, the standard pooled interval may not have its advertised coverage. A large \(m\) reduces Monte Carlo noise in the computation; it cannot repair bias from a poor model or an unsupported missingness assumption.
The interval should therefore be read as an uncertainty statement under the specified approach, not as a guarantee that all sources of uncertainty have been captured. The number of imputations, the within- and between-imputation components, and the assumptions underlying the procedure each matter for different reasons.
Common Mistakes
- Using only the average within-imputation variance. This omits the between-imputation contribution. Use \(T=\bar{U}+(1+1/m)B\), not just \(\bar{U}\).
- Thinking more imputations remove all missing-data uncertainty. The finite adjustment shrinks with \(m\), but \(B\) remains in the limiting pooled variance.
- Calling a confidence interval a probability statement about the fixed coefficient. A 95% procedure has 95% coverage in repeated applications under its assumptions; it does not assign a 95% probability to the coefficient being in the interval already calculated.
- Treating a narrow interval as proof the imputation assumptions hold. Precision under a model does not establish that the model or missingness assumptions are adequate.
- Forgetting context and units. A regression coefficient interval should identify the outcome and predictor units, such as health-score points per predictor unit.
Check Your Understanding
Answer these questions using the pooled-uncertainty ideas from this tutorial.
- In a pooled analysis, what does \(\bar{U}\) measure, and what does \(B\) measure?
- If \(\bar{U}=0.20\), \(B=0.05\), and \(m=4\), calculate \(T\) and the pooled standard error.
- As the number of imputations increases, which part of \(T=\bar{U}+(1+1/m)B\) shrinks, and which between-imputation contribution remains?
- What does a 95% pooled confidence interval mean under the model and imputation assumptions?
- Why can a large number of imputations fail to fix a poorly specified imputation model?