Pool Estimates Across Imputations and Scenarios

Why Pooling Must Follow the Analysis Structure

Multiple imputation produces several completed versions of one incomplete dataset. The same treatment contrast is estimated in each version, and the estimates differ because the missing values are uncertain. Rubin’s rules combine that variation with the sampling variance from each completed-data analysis. In a sensitivity analysis, however, imputations may also be generated under different assumptions about the unobserved outcomes. Those assumptions define distinct scenarios, not extra draws from one common imputation procedure.

The practical rule is to pool within each scenario and then compare the resulting scenario-specific estimates. A scenario might represent a baseline missing-at-random analysis, a delta adjustment, or a reference-based imputation assumption. If two scenarios imply different distributions for the missing outcomes, combining their estimates as though they came from one imputation set would blur the distinction the sensitivity analysis is intended to examine.

Calculate the Contrast Before Pooling

The analysis performed in each completed dataset must estimate the same contrast, on the same scale, and for the same target population. For a simple two-arm comparison, define the contrast as treatment minus control. In completed dataset \(m\), if the outcome is quantitative and the groups are independent, one possible estimate and its variance are

Here the standard error is \(\sqrt{U_m}\). With an adjusted regression analysis, \(Q_m\) is the specified model contrast and \(U_m\) is its estimated variance, calculated using the fitted model’s covariance matrix. The chosen analysis and variance estimator should be appropriate to the outcome and design; Rubin’s rules do not repair a misspecified or inappropriate completed-data analysis.

A short justification explains the structure of the pooling formula. If the imputation model and analysis are appropriate, uncertainty in the pooled estimate has two components: average sampling uncertainty within completed datasets, represented by \(\bar{U}\), and uncertainty about missing values, represented by \(B\). Averaging \(M\) estimates reduces the sampling contribution from between-imputation variation to approximately \(B/M\), while the uncertainty associated with the missing-data distribution contributes \(B\). Thus the additional component is \(B+B/M\), giving

This decomposition is the basis of Rubin’s total variance. The usual large-sample interval uses \(\bar{Q}\pm1.96\sqrt{T}\). When inference requires a finite-sample reference distribution, degrees of freedom should be chosen using an appropriate multiple-imputation adjustment and the complete-data degrees of freedom; the simple Rubin degrees of freedom are \((M-1)(1+1/r)^2\), where \(r=(1+1/M)B/\bar{U}\). In the examples below, large-sample normal intervals are used for clarity.

Keep Sensitivity Scenarios Separate

Indexing estimates by both scenario and imputation makes the correct workflow explicit. Apply Rubin’s rules to \(Q_{s1},\ldots,Q_{sM}\) and \(U_{s1},\ldots,U_{sM}\) for a fixed \(s\). Repeat for each scenario. The resulting estimates can be compared in a table or plot, but the scenario index is not an additional imputation index.

The distinction is substantive, not merely computational. Within a scenario, the imputed datasets represent uncertainty about missing values under a defined set of assumptions. Across scenarios, the assumptions themselves change. An average across scenarios would require an explicit and defensible model assigning probabilities or weights to those assumptions; Rubin’s rules do not supply such a model.

Model-Based Contrasts Use the Same Pooling Rules

The contrast need not be a difference between raw means. If a completed-data model estimates a coefficient vector \(\hat{\boldsymbol{\beta}}_m\) with covariance matrix \(\widehat{\mathbf{V}}_m\), a linear contrast with specified coefficient vector \(\mathbf{c}\) is

The contrast definition and coefficient coding must remain consistent in every imputation. Pool the resulting \(Q_m\) and \(U_m\) within the scenario using the same formulas. This approach applies, for example, to an adjusted treatment contrast formed from model coefficients, provided the model and estimand are appropriate.

Diagnostics, Alternatives, and Limitations

Before reporting pooled results, check that the same estimand, units, contrast direction, model specification, and variance method were used for every imputation within each scenario. Confirm that \(M\) estimates and their variances are available for that scenario. Review whether the estimates and standard errors are plausible and whether any imputation failed; silently dropping failed fits can change the analysis and conceal instability. A very large \(B\) relative to \(\bar{U}\) signals substantial variation across imputations and may lead to wide uncertainty. A very small \(B\) is not proof that the missing-data assumptions are correct.

Rubin’s rules require more than a list of estimates: the imputations should be generated appropriately under the stated scenario, and the completed-data analysis should target the same quantity as the imputation procedure. The simple large-sample interval may be inadequate with few imputations, limited complete-data degrees of freedom, or strongly non-normal estimates. Some estimates, such as ratios, may need pooling on a suitable transformed scale with results transformed back for presentation. These choices should be specified rather than improvised after looking at results.

Common alternatives have different properties. Complete-case analysis discards incomplete records and does not use the imputed datasets. Single imputation supplies one completed dataset but does not represent between-imputation uncertainty, so treating its standard error as if it included all missing-data uncertainty is generally misleading. Pooling across sensitivity scenarios is not an alternative Rubin’s-rules calculation: it changes the question and needs an explicit model for scenario uncertainty. The appropriate workflow is to present one pooled estimate and uncertainty measure per scenario, making the assumptions visible alongside the results.