Pooling Integrated Estimates Across Imputations

Why Pooling Must Include More Than an Average

Suppose a multi-omics analysis integrates available modalities to estimate one association, but some modality blocks are missing for some subjects. Multiple imputation creates \(M\) completed data sets, and the same integrated analysis is performed on each. The resulting estimates need not agree: some variation reflects ordinary sampling uncertainty, and some reflects uncertainty about the missing values. A useful pooled result must account for both.

The earlier material on multiple imputation addresses constructing completed data sets and preserving cross-omics dependence. Here, the focus is inference after that step: combining a scalar estimate and its variance across imputations. The central rule is that the average of the estimates is not, by itself, a complete uncertainty calculation.

All \(Q_l\) must estimate the same quantity, using the same scale and compatible analysis. For example, an estimated regression coefficient in each imputation is a suitable scalar quantity if the target coefficient and its units are unchanged. \(U_l\) is the variance estimate for that coefficient in imputation \(l\), not its standard error: if the reported standard error is \(s_l\), then \(U_l=s_l^2\).

Why the Total Variance Has Two Parts

The average within-imputation variance \(\bar{U}\) captures sampling uncertainty that would remain if the completed data set were known. The estimates also vary across imputations. The sample variance \(B\) measures that additional variation due to uncertainty about missing values. Simply adding \(\bar{U}+B\) would describe the large-\(M\) variance component but omit the Monte Carlo contribution from estimating the pooled result with only \(M\) imputations.

A variance-decomposition argument motivates the rule. Conditional on the observed data, uncertainty in the target can be separated into expected complete-data variance and variability in the target’s conditional expectation over the missing values. The first component is estimated by \(\bar{U}\); the second is estimated by \(B\). With a finite number \(M\) of imputations, the estimated average itself has additional simulation variance approximately \(B/M\). Adding these pieces gives \(\bar{U}+B+B/M\), which is Rubin’s \(T\). This is the standard multiple-imputation variance rule, not a claim that the imputation-specific estimates are independent samples from the population.

This calculation relies on more than having several completed data sets. The imputation procedure must be proper: it should represent uncertainty in missing values and relevant model parameters, rather than repeatedly filling in values as if imputation parameters were known exactly. The complete-data estimator should have a reasonable approximately normal sampling distribution, and each \(U_l\) should estimate its variance appropriately. The analysis and imputation models must be suitably specified for the estimand. For likelihood-based inference under an ignorable missingness mechanism, the usual conditions include missing at random conditional on included information and distinct missingness and data-model parameters. If these conditions are not credible, the pooled standard error does not resolve the resulting model uncertainty.

Degrees of Freedom and Confidence Intervals

Because \(B\) is estimated from only \(M\) values, the pooled estimate does not generally use an ordinary normal critical value. A standard large-complete-data-sample approximation uses Rubin’s degrees of freedom. Define \(r=(1+1/M)B/\bar{U}\), the relative increase in variance due to missingness. Then the degrees of freedom are:

For a two-sided \(100(1-\alpha)\%\) interval, use \(\bar{Q}\pm t_{1-\alpha/2,df}\sqrt{T}\). The degrees of freedom are a function of the relative within- and between-imputation uncertainty; they are not the number of subjects, nor should they be set equal to the number of imputations. When \(B\) is zero, this expression has a limiting value of infinity, but a zero observed between-imputation variance can also reflect too few imputations or a problematic imputation procedure.

If the complete-data analysis has limited degrees of freedom, the large-sample formula may overstate the available information. A small-sample adjustment, such as the Barnard–Rubin adjustment, combines the Rubin degrees of freedom with a complete-data degrees-of-freedom limit. In that setting, use the adjustment appropriate to the analysis rather than treating the large-sample formula as universally exact. When \(B\) is substantial relative to \(\bar{U}\), increasing \(M\) can make the estimated between-imputation component more stable; it does not remove genuine uncertainty from missing data.

Several Estimates and Their Covariances

An integrated analysis may produce a vector of estimates, such as coefficients for two modality-specific predictors. Pool each component’s mean estimate, but also retain their dependence when making joint inferences. Let \(\mathbf{Q}_l\) be the estimate vector and \(\mathbf{U}_l\) its within-imputation covariance matrix. Define \(\bar{\mathbf{Q}}=M^{-1}\sum_l\mathbf{Q}_l\), \(\bar{\mathbf{U}}=M^{-1}\sum_l\mathbf{U}_l\), and \(\mathbf{B}=(M-1)^{-1}\sum_l(\mathbf{Q}_l-\bar{\mathbf{Q}})(\mathbf{Q}_l-\bar{\mathbf{Q}})^\mathsf{T}\). The pooled covariance matrix is \(\mathbf{T}=\bar{\mathbf{U}}+(1+1/M)\mathbf{B}\). Its diagonal entries give total variances for individual coefficients; its off-diagonal entries matter for contrasts and joint tests.

Diagnostics, Alternatives, and Common Pitfalls

Pooling is meaningful only if the estimates are comparable. Before interpreting the result, check that every completed-data analysis used the same estimand, model specification, coding, and measurement scale. Review the \(Q_l\) values and \(U_l\) values for transcription or fitting failures. Between-imputation variation is expected, but a striking outlier, a highly unstable estimate, or implausibly identical estimates across all imputations can signal problems. A very large \(B\) relative to \(\bar{U}\) warns that missing-data uncertainty is influential; a very small \(B\) is not proof that the missingness problem is negligible.

Do not pool confidence-interval endpoints or average p-values. The pooling rules apply to estimates and their variance estimates; inferential intervals and tests are then calculated from the pooled quantities. Likewise, do not average standard errors in place of variances: the within-imputation component is the average of \(U_l\), which are squared standard errors. If the estimand is nonlinear, pool on a scale where the estimate is approximately normal, then transform the estimate and interval back if needed. For example, a ratio estimate is often analysed on the log scale, with the interval endpoints exponentiated afterward.

Rubin’s rules are the standard approach for combining estimates from proper multiple imputations when the estimand and variance estimates meet the conditions described above. They are not a substitute for a well-specified imputation model or a remedy for an implausible missingness assumption. A single analysis of one filled-in data set typically omits imputation uncertainty. A complete-case analysis answers a potentially different question and may lose information; it is not another way of applying the pooling formulas. Other inferential approaches may be appropriate for a different model or estimand, but they require their own assumptions and uncertainty calculations.

• Wrong: “The pooled standard error is the average of the five standard errors.” Correction: Average the within-imputation variances, add the adjusted between-imputation variance, and take the square root.
• Wrong: “The estimates differ, so I should choose the imputation closest to their average.” Correction: Use all imputations in \(\bar{Q}\), \(B\), and \(T\); the differences contribute to uncertainty.
• Wrong: “A large number of subjects means I can ignore the number of imputations.” Correction: The number of imputations affects the finite-\(M\) adjustment and the stability of \(B\), even in a large study.