Match the method to the question
After multiple imputation, the right pooling procedure depends on what the analysis is meant to report. A single regression coefficient, a contrast between two coefficients, and a joint test that several coefficients are zero are different targets. They may use related information from the completed data sets, but they do not automatically call for the same pooling calculation.
A useful first distinction is between estimation and testing. For a scalar estimate, Rubin’s rules pool the estimates and their variances. A linear contrast can also be pooled as a scalar, provided its variance is calculated from the full within-imputation covariance matrix. A joint hypothesis usually needs a multivariate test. Depending on what each completed-data analysis provides, that may be D1, D2, or D3. If the target is more naturally expressed on a transformed scale, pool there and then transform the result back.
These rules require each \(Q_j\) to estimate the same estimand on the same scale, with its uncertainty estimated appropriately in each completed data set. Valid MI inference also depends on suitable imputation and analysis models; pooling cannot repair a misspecified analysis, an unsuitable imputation model, or incompatible targets. When using a test procedure, the completed-data test must also match the same null hypothesis across imputations.
Scalar estimates and linear contrasts
For a scalar target, the pooled estimate is \(\bar Q\), and its standard error is \(\sqrt{T}\). A commonly used large-sample Rubin degrees-of-freedom approximation is based on \(r=(1+1/m)B/\bar U\) and \(\nu=(m-1)(1+1/r)^2\). A confidence interval uses a \(t\) critical value with those degrees of freedom. If complete-data degrees of freedom are relevant, use a small-sample adjustment rather than automatically treating this large-sample approximation as exact.
Worked Example: Pooling One Regression Coefficient
Suppose four hypothetical imputations produce the same slope estimate, in outcome units per one-unit increase in a predictor, and its estimated variance:
| Imputation \(j\) | \(Q_j\) | \(U_j\) |
|---|---|---|
| 1 | 1.20 | 0.040 |
| 2 | 1.10 | 0.050 |
| 3 | 1.30 | 0.040 |
| 4 | 1.00 | 0.050 |
The pooled estimate is \(\bar Q=(1.20+1.10+1.30+1.00)/4=1.15\). The average within-imputation variance is \(\bar U=(0.040+0.050+0.040+0.050)/4=0.0450\). The between-imputation variance is
Thus \(T=0.0450+1.25(0.01667)=0.06583\), giving \(SE=\sqrt{0.06583}=0.2566\). Here \(r=1.25(0.01667)/0.0450=0.4630\), so \(\nu=3(1+1/0.4630)^2=29.95\). Using \(t_{0.975,29.95}\approx2.045\), the 95% interval is \(1.15\pm2.045(0.2566)=[0.625,1.675]\). In this hypothetical analysis, the estimated slope is 1.15 outcome units per predictor unit; the interval describes uncertainty in that slope under the analysis and MI assumptions.
A linear contrast of a vector of coefficients is still a scalar target. If \(\hat\beta_j\) is the coefficient vector and \(U_j\) its covariance matrix in imputation \(j\), a contrast defined by a fixed vector \(c\) is \(Q_j=c'\hat\beta_j\), with within-imputation variance \(U_j^{(c)}=c'U_jc\). Pool these \(Q_j\) and \(U_j^{(c)}\) using scalar Rubin rules. This uses the covariance between coefficients; subtracting or combining standard errors separately does not.
Worked Example: A Difference Between Two Coefficients
Suppose a hypothetical model estimates two coefficients, \(\beta_1\) and \(\beta_2\), in three imputations. The question concerns their difference, \(\beta_1-\beta_2\), so \(c=(1,-1)'\). The imputation-specific coefficient vectors are \((2.0,0.5)'\), \((1.8,0.4)'\), and \((2.2,0.7)'\). Their contrast estimates are \(1.5,1.4,1.5\). Suppose the corresponding contrast variances, calculated as \(c'U_jc\), are \(0.090,0.100,0.090\).
The pooled contrast is \(\bar Q=(1.5+1.4+1.5)/3=1.4667\). Its average within-imputation variance is \(\bar U=(0.090+0.100+0.090)/3=0.09333\), and \(B=[(1.5-1.4667)^2+(1.4-1.4667)^2+(1.5-1.4667)^2]/2=0.003333\). Therefore \(T=0.09333+(1+1/3)(0.003333)=0.09778\), and \(SE=\sqrt{0.09778}=0.3127\). The target is the difference in coefficients, estimated here as 1.4667 coefficient units. Its variance must include the covariance of the two coefficients within each imputation.
Joint hypotheses: D1, D2, and D3
For a joint hypothesis such as \(H_0:\beta_1=\beta_2=0\), the target is not one scalar coefficient. The main choice among D1, D2, and D3 is the type of information available from each completed-data analysis.
| Method | Information it uses | Typical setting |
|---|---|---|
| D1 | Parameter estimates and their covariance matrices | A multivariate Wald test of a joint restriction |
| D2 | Completed-data test statistics for the same null, with their degrees of freedom | Test output is available but pooled parameter covariance information is not |
| D3 | Likelihood-ratio information from nested models | Likelihood-based comparisons are available and their assumptions are appropriate |
D1 is the natural extension of pooling estimates and their covariance information. For a \(k\)-dimensional parameter vector, let \(\bar Q\) be its pooled estimate and \(\bar U\) and \(B\) its within- and between-imputation covariance matrices. For the null value \(Q_0\), set \(T=\bar U+(1+1/m)B\). A commonly used D1 approximation forms a Wald \(F\) statistic using \(r_1=(1+1/m)\operatorname{tr}(B\bar U^{-1})/k\):
The reference distribution is approximately \(F_{k,\nu_2}\). This is an approximation for MI inference, not the ordinary complete-data Wald test applied without adjustment.
Worked Example: A Two-Parameter D1 Test
Consider hypothetical pooled estimates \(\bar Q=(0.4,0.3)'\) for a joint null \(Q_0=(0,0)'\), with \(m=4\), \(\bar U=\begin{pmatrix}0.040&0\\0&0.040\end{pmatrix}\), and \(B=\begin{pmatrix}0.010&0\\0&0.010\end{pmatrix}\). The total covariance is \(T=\bar U+1.25B=\begin{pmatrix}0.0525&0\\0&0.0525\end{pmatrix}\). Since both diagonal entries are equal, the quadratic form is \(0.4^2/0.0525+0.3^2/0.0525=4.762\), so \(F_{\mathrm{D1}}=4.762/2=2.381\).
Here \(r_1=1.25(0.010/0.040+0.010/0.040)/2=0.3125\), and \(\nu_2=4+3(1+1/0.3125)^2=56.92\). The approximate reference is \(F_{2,56.92}\), giving a p-value of about 0.102. At the 0.05 significance level, this test does not provide sufficient evidence against the joint null. This is not evidence that both coefficients equal zero.
D2 combines completed-data test statistics rather than pooling coefficient vectors. In a common formulation, let \(d_j\) be the \(j\)th completed-data chi-square statistic for the same \(k\)-degree-of-freedom null, and let \(\bar d=m^{-1}\sum_jd_j\). Define \(r_2=(1+1/m)s_d^2/\bar d\), where \(s_d^2=(m-1)^{-1}\sum_j(d_j-\bar d)^2\). The approximate pooled statistic is \(D2=\bar d/[k(1+r_2)]\), compared with \(F_{k,\nu_2}\), where \(\nu_2=(m-1)(1+1/r_2)^2\). Use a procedure appropriate to the test-statistic scale and degrees of freedom; do not substitute averaged p-values.
Worked Example: Combining Test Statistics With D2
Suppose three hypothetical completed-data analyses yield chi-square statistics \(3.2,4.0,2.8\) for the same two-degree-of-freedom null. Then \(\bar d=(3.2+4.0+2.8)/3=3.333\). The sample variance is \(s_d^2=[(3.2-3.333)^2+(4.0-3.333)^2+(2.8-3.333)^2]/2=0.3733\). Thus \(r_2=(1+1/3)(0.3733)/3.333=0.1493\), and \(D2=3.333/[2(1+0.1493)]=1.450\). The denominator degrees of freedom are \(\nu_2=2(1+1/0.1493)^2=118.5\). The approximate p-value from \(F_{2,118.5}\) is 0.238, so the test does not reject this joint null at the 0.05 level.
D3 is a likelihood-ratio approach for comparing nested models. It uses the imputation-specific likelihood-ratio information for the same restricted-versus-unrestricted comparison and an MI-specific calibration to obtain a pooled test and reference distribution. It is appropriate only when the models, likelihoods, and regularity conditions support that likelihood-based comparison. Because its calibration is not obtained by averaging likelihood-ratio p-values, use an implementation that explicitly provides D3 rather than applying a complete-data likelihood-ratio reference distribution to an average.
When to pool on a transformed scale
Some estimates are better behaved, or more naturally interpreted, on a transformed scale. For an odds ratio, for example, the log odds ratio can be approximately normally distributed even when the odds ratio is asymmetric. Pool the log-scale estimates and their variances, form the interval on that scale, and exponentiate the pooled estimate and both interval endpoints. This is different from pooling odds ratios directly or transforming each imputation’s confidence limits before pooling.
Worked Example: Pooling a Log Odds Ratio
Suppose three hypothetical imputations give log odds ratios \(0.40,0.50,0.30\) with variances \(0.040,0.050,0.040\). These are all estimates of the same log odds ratio. The pooled log odds ratio is \(\bar Q=0.40\). The average within-imputation variance is \(\bar U=0.04333\), and \(B=[(0.40-0.40)^2+(0.50-0.40)^2+(0.30-0.40)^2]/2=0.0100\). Hence \(T=0.04333+(1+1/3)(0.0100)=0.05667\), so \(SE=0.2380\).
The estimated odds ratio is \(\exp(0.40)=1.492\). Using the large-sample Rubin degrees of freedom, \(r=1.333(0.0100)/0.04333=0.3077\) and \(\nu=2(1+1/0.3077)^2=36.13\). With \(t_{0.975,36.13}\approx2.028\), the log-scale interval is \(0.40\pm2.028(0.2380)=[-0.0827,0.8827]\). Exponentiating gives an odds-ratio interval of approximately \([0.921,2.417]\). The interval includes 1, so it does not exclude the null odds ratio of 1 at the two-sided 0.05 level.
Selection checks and common pitfalls
Decide whether the question concerns one scalar estimate, a specified contrast, a joint restriction, or a parameter expressed on a transformed scale.
Scalar Rubin pooling needs estimates and variances; D1 needs estimates and covariance matrices; D2 needs compatible completed-data test statistics; D3 needs suitable nested-model likelihood information.
Confirm that every imputation estimates the same quantity and that the selected test or transformation is appropriate for the analysis model.
Give the estimate or test statistic, the pooling method, uncertainty measure or reference degrees of freedom, and an interpretation in the original scientific units or scale.
A common mistake is to select a method because its output is easiest to obtain, then treat it as interchangeable with the procedure that matches the scientific question. For example, averaging the p-values from separate imputations does not pool either an estimate or a valid MI test. Likewise, a D1 test cannot be reconstructed from marginal standard errors if the covariance between coefficients is required but unavailable. If essential information is missing, report that limitation or obtain the needed output; do not replace it with an incompatible calculation.
Diagnostics include checking that all imputations use the same model, parameterization, contrast, and null hypothesis; confirming that covariance matrices are available and valid for D1; and verifying that a D2 procedure receives statistics on the scale and with the degrees of freedom it expects. For transformed pooling, inspect whether the chosen transformation is defined for all estimates and whether the final back-transformed interval is interpretable. These procedures do not eliminate sensitivity to the imputation model, small numbers of imputations, poor model fit, or weak information about the target.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- You have a coefficient vector and its covariance matrix in every imputation, and want to test two coefficients jointly. Which method is a natural choice, and what key information does it use?
- You want the difference between two regression coefficients. How should its within-imputation variance be calculated?
- Three completed-data analyses provide test statistics and p-values, but no pooled covariance matrix. Is averaging their p-values a suitable MI test?
- For an odds ratio, why might you pool on the log scale and exponentiate the interval endpoints afterward?
Solutions. (1) D1 is a natural choice: it uses the pooled parameter estimates and within- and between-imputation covariance matrices to form a joint Wald test. (2) For contrast vector \(c=(1,-1)'\), calculate \(c'U_jc\) in each imputation, then pool the resulting contrast estimates and variances as a scalar. (3) No. Averaged p-values are not a valid replacement for an MI test; D2 may be appropriate if the test statistics and their degrees of freedom meet that method’s requirements. (4) The log odds ratio is often more suitable for approximately symmetric inference. Pool the log-scale estimates and variances, form the interval there, and exponentiate the estimate and interval limits to report odds-ratio-scale results.