Why Unequal Standard Errors Need Careful Pooling
A multiple-imputation analysis produces an estimate and an estimated variance from each completed data set. Even when every analysis targets the same scalar parameter, the estimates and their standard errors need not be identical across imputations. Missing values have been filled in differently, and the completed data sets can therefore yield different estimates and different measures of sampling uncertainty.
Pooling must account for both sources of variation. The within-imputation variances summarize sampling uncertainty within the completed data sets; the between-imputation variance summarizes how much the estimates vary across imputations. Rubin’s total-variance formula combines these components. This tutorial works through that calculation for five imputations with unequal standard errors. It does not calculate degrees of freedom, confidence intervals, or tests.
If the reported standard error from imputation \(j\) is \(SE_j\), its variance is \(U_j=SE_j^2\). The standard errors themselves are not the quantities averaged to obtain \(\bar U\). All estimates must refer to the same parameter, on the same scale, under the same analysis specification. For example, five estimates of a regression coefficient for the same predictor and outcome can be pooled; estimates of different coefficients cannot.
What the Calculation Requires
The arithmetic in Rubin’s rules is defined once the estimates and variances are supplied. Its inferential interpretation relies on conditions about how those values were obtained. In particular, each completed-data analysis should use the same estimand and analysis method, and each \(U_j\) should estimate the sampling variance of \(Q_j\) under that analysis. The multiple imputations should be generated in a way that appropriately represents uncertainty about missing values, rather than simply filling in fixed predictions.
For the usual large-sample scalar interpretation, estimates should be sufficiently well behaved that their sampling distributions are approximately normal, with variance estimates that are reasonably calibrated. These conditions do not make the five estimates identical, nor do they require equal \(U_j\). Unequal standard errors are allowed; each completed-data variance contributes to the average \(\bar U\).
A short justification of the between-imputation calculation is useful. The \(Q_j\) values are the estimates whose dispersion across imputations is being summarized. Their deviations from their own average add to zero, and their squared deviations measure spread in the parameter’s units squared. Dividing that sum by \(M-1\) gives the sample variance of the \(M\) estimates:
The total variance then adds the mean within-imputation variance to the between-imputation component, with the factor \(1+1/M\) specified by Rubin’s rules. The pooled standard error is the square root of this total variance, so it returns to the original parameter’s units. The calculation uses variances throughout until that final square root.
A Five-Imputation Calculation
Worked Example: Pooling Five Estimates With Unequal Standard Errors
Suppose five hypothetical imputations produce estimates of a mean change in a clinical score, measured in points. The reported standard errors are also in points.
| Imputation \(j\) | Estimate \(Q_j\) (points) | Standard error \(SE_j\) (points) | Variance \(U_j=SE_j^2\) (points squared) |
|---|---|---|---|
| 1 | 12.0 | 0.8 | 0.64 |
| 2 | 13.2 | 1.0 | 1.00 |
| 3 | 11.5 | 0.9 | 0.81 |
| 4 | 12.8 | 1.2 | 1.44 |
| 5 | 10.5 | 0.7 | 0.49 |
1. Pooled estimate. Add the five estimates and divide by five:
2. Mean within-imputation variance. Square each standard error, as shown in the table, then average the variances:
3. Between-imputation variance. The deviations from \(\bar Q=12.0\) are \(0.0,\ 1.2,\ -0.5,\ 0.8,\) and \(-1.5\) points. Their squared deviations sum to \(0+1.44+0.25+0.64+2.25=4.58\) points squared. Since \(M-1=4\):
4. Total variance and pooled standard error. Here \(M=5\), so \(1+1/M=1.2\):
The pooled point estimate is 12.0 points, and its pooled standard error is 1.5 points. The standard error summarizes the estimated uncertainty in the pooled estimate; it is not a measure of how far every individual imputation estimate lies from 12.0.
Two More Checks on the Arithmetic
Worked Example: Pooling a Regression Coefficient
In a hypothetical analysis, five imputations give a regression coefficient for exposure of \(4.1,\ 4.4,\ 3.8,\ 4.3,\) and \(4.0\) mmHg per exposure unit. Their standard errors are \(0.5,\ 0.7,\ 0.4,\ 0.6,\) and \(0.8\) mmHg per exposure unit. The same coefficient, outcome, and model are used for all five fits.
The pooled coefficient is
The within-imputation variances are \(0.25,\ 0.49,\ 0.16,\ 0.36,\) and \(0.64\) in squared coefficient units. Therefore,
The deviations from 4.12 are \(-0.02,\ 0.28,\ -0.32,\ 0.18,\) and \(-0.12\). Their squared sum is \(0.0004+0.0784+0.1024+0.0324+0.0144=0.2280\). Thus \(B=0.2280/4=0.0570\) in squared coefficient units. Combining the components gives
The pooled coefficient is 4.12 mmHg per exposure unit, with pooled standard error 0.6696 mmHg per exposure unit. This is an association estimate under the specified model; the calculation alone does not establish a causal effect.
Worked Example: Checking the Variance Scale
Suppose five hypothetical imputations estimate a proportion, giving \(Q_j=0.20,\ 0.22,\ 0.18,\ 0.21,\ 0.19\). Their standard errors are \(0.04,\ 0.08,\ 0.05,\ 0.10,\ 0.03\), in proportion units.
The estimate average is \(1.00/5=0.20\). Squaring the standard errors gives \(0.0016,\ 0.0064,\ 0.0025,\ 0.0100,\) and \(0.0009\) proportion-squared. Hence
The deviations from 0.20 are \(0,\ 0.02,\ -0.02,\ 0.01,\) and \(-0.01\). The squared deviations sum to \(0.0004+0.0004+0.0001+0.0001=0.0010\), so \(B=0.0010/4=0.00025\). Therefore,
The pooled estimate is 0.20, or 20% when expressed as a percentage, and the pooled standard error is 0.06768 in proportion units (about 6.768 percentage points). The variance calculations stay on the proportion scale until the final square root.
Diagnostics, Alternatives, and Limits
Several quick checks can catch errors before a pooled result is reported:
- Confirm that each \(Q_j\) estimates the same scalar parameter and that every \(U_j\) is its variance estimate, not its standard error.
- Check that \(U_j\) is nonnegative and has squared parameter units, while \(Q_j\) and \(\sqrt T\) have parameter units.
- Recalculate \(\bar Q\) and the deviations from it; their sum should be zero apart from rounding.
- Use \(M-1\) in the sample variance \(B\), and use the actual number of imputations in \(1+1/M\).
- Inspect unusually different \(Q_j\) or \(U_j\) values. They may be genuine, but can also flag inconsistent model fitting, unstable estimates, or a data-processing error.
A common mistake is to average standard errors and call the result the pooled standard error. That is not Rubin’s calculation: the within-imputation component is formed by averaging variances, and the between-imputation component must also be included. Likewise, treating the \(M\) completed data sets as if they were independent new samples, or simply reporting the result from one completed data set, does not account for uncertainty in the missing values.
Inverse-variance weighting is another tempting alternative when standard errors differ. Rubin’s scalar pooled estimate is the arithmetic mean of the \(Q_j\) values, not an inverse-variance-weighted mean. The estimates are repeated analyses of one target using multiple imputations, and their variation is used alongside the within-imputation variances. Unequal standard errors therefore change \(\bar U\), but do not change the definition of \(\bar Q\).
This pooling calculation has limits. It cannot repair an inappropriate estimand, invalid imputation model, or incompatible analyses across imputations. It is also a scalar procedure: when the target is a vector of estimates, their covariance must be considered rather than pooling each component as though it were unrelated. Finally, the pooled standard error by itself is not a confidence interval or a test; those require additional inferential calculations.
Verify that all imputations estimate the same parameter using the same model and scale.
Average the estimates, square and average the standard errors, and calculate the sample variance of the estimates.
Calculate \(T=\bar U+(1+1/M)B\), then take \(\sqrt T\); the result must be in the parameter’s units.
Check Your Understanding
Answer each question, then compare your work with the solutions below.
- Five scalar estimates are \(2,\ 3,\ 1,\ 2,\ 2\), and their within-imputation variances are \(1,\ 4,\ 1,\ 4,\ 1\). Calculate \(\bar Q,\ \bar U,\ B,\ T,\) and the pooled standard error.
- Why must reported standard errors be squared before calculating \(\bar U\)?
- For the estimates in Question 1, what is the sum of deviations from \(\bar Q\), and what does that check?
- A colleague pools five estimates of one coefficient and five estimates of a different coefficient together as ten values. What is wrong with this calculation?
Solutions. (1) \(\bar Q=(2+3+1+2+2)/5=2\). The mean variance is \(\bar U=(1+4+1+4+1)/5=2.2\). Squared deviations from 2 sum to \(0+1+1+0+0=2\), so \(B=2/4=0.5\). Then \(T=2.2+1.2(0.5)=2.8\), and the pooled standard error is \(\sqrt{2.8}=1.673\). (2) A standard error has parameter units, whereas a variance has squared parameter units; Rubin’s within component averages variances. (3) The deviations are \(0,1,-1,0,0\), whose sum is 0. This checks that deviations were taken from their arithmetic mean. (4) The two coefficients are different estimands. Pooling them together would mix unlike quantities, so each parameter must be pooled separately across its corresponding imputations.