The Pitfall Of Pooling Standard Errors Instead Of Variances

Why the Order of Pooling Matters

A standard error is the square root of a variance. That relationship can make it tempting to average the standard errors from completed data sets and treat the result as the pooled standard error. But multiple imputation introduces two sources of uncertainty: sampling uncertainty within each completed-data analysis and variation in estimates across imputations. Rubin’s rules combine these sources on the variance scale; the pooled standard error is then the square root of the resulting total variance.

Averaging standard errors is not just a different route to the same result. It averages square roots rather than variances, and it does not account for between-imputation variation. The error can be substantial when the estimates vary across imputations, and it can remain even when those estimates are identical if the complete-data standard errors differ.

The multiplier \(1+1/M\) accounts for the fact that the missing-data uncertainty is estimated from a finite number of imputations. The term \(\bar{U}\) is the average estimated sampling variance within completed data sets. The term \(B\) measures how much the estimates differ across them. Both \(U_j\) and \(T\) are variances, so their units are the square of the units of \(\hat{q}_j\); their square roots are on the estimate’s original scale.

These rules presume that each completed-data analysis estimates the same quantity on the same scale, that \(U_j\) is an appropriate estimate of its complete-data sampling variance, and that the imputations adequately represent uncertainty about missing values. Rubin’s variance formula is an inferential approximation under the usual multiple-imputation conditions; it does not repair a poorly specified imputation or analysis model. For the arithmetic comparison here, the key requirements are that the estimates and variances refer to a common scalar estimand and are reported on a consistent scale.

Why Standard Errors Cannot Be Pooled Directly

Write the complete-data standard error in imputation \(j\) as \(s_j=\sqrt{U_j}\). A tempting shortcut is to calculate \(M^{-1}\sum_j s_j\) and call it the pooled standard error. Even if there were no variation in the estimates, this is generally not equal to \(\sqrt{\bar{U}}\), because taking a square root and taking an average do not commute. More importantly, the shortcut omits \(B\).

The first inequality follows from the concavity of the square-root function: for nonnegative \(U_j\), Jensen’s inequality gives the average of their square roots as no greater than the square root of their average. The second follows because \(B\geq 0\) and \(1+1/M>0\). Thus, for this scalar Rubin pooling calculation, the average complete-data standard error cannot exceed the pooled standard error. Equality requires both equal \(U_j\) values and \(B=0\).

This result quantifies the direction of the shortcut’s error, but not its size: the discrepancy can be small or large depending on the within-imputation variances and the spread of the estimates. Squaring the average standard error does not fix the problem either. In general, \((M^{-1}\sum_j\sqrt{U_j})^2\) is not \(\bar{U}\), and it still contains no between-imputation component.

A Reliable Calculation and Its Checks

The safe order is to retain each complete-data variance, pool those variances with Rubin’s rules, and take a square root only once, at the end. The following sequence also helps catch scale and arithmetic errors.

If a reported pooled standard error is below the average of the complete-data standard errors in this scalar setting, inspect the calculation: it conflicts with the inequalities above. Common causes include omitting \(B\), taking the square root too early, using \(B\) with a missing or incorrect finite-\(M\) multiplier, or mixing estimates and variances on different scales. A negative variance component or a variance with units that do not match the squared estimate units also signals an error.

Limits and Common Pitfalls

The formulas here concern a scalar estimate with compatible variance estimates. They are not a license to pool standard errors for a multicomponent parameter as if each component were independent; joint inference uses the relevant covariance structure. Nor does a correctly computed pooled standard error guarantee valid inference if the imputation model, analysis model, or common-estimand requirement is not appropriate. Rubin’s rules also involve finite-sample degrees of freedom for some inferential procedures; the pooled standard error alone does not supply every part of an interval or test.

• Wrong: “The standard error is already on the estimate’s scale, so averaging the imputation-specific standard errors is the natural pooling rule.” Correction: Standard errors are square roots of variances. Pool \(U_j\) and the between-imputation component first, then take one square root.
• Wrong: “The point estimates are close, so the between-imputation component can be dropped.” Correction: Calculate \(B\) from the estimates. Even when \(B=0\), averaging standard errors can differ from \(\sqrt{\bar{U}}\) if the \(U_j\) vary.
• Wrong: “The average of the complete-data variances is the full multiple-imputation variance.” Correction: \(\bar{U}\) is only the within-imputation component; Rubin’s total variance also includes \((1+1/M)B\).