Pooling Estimates Across Delta-Adjusted Imputations

Why Pool Separately at Each Delta?

A delta-adjusted sensitivity analysis repeats imputation under alternative assumptions about missing values. At a given delta value, the completed data sets represent repeated draws under one specified adjustment. Rubin’s rules combine the estimates from those completed data sets while accounting for uncertainty within and between imputations. The calculation must be repeated at every delta value in the sensitivity grid.

The key distinction is between two kinds of variation. Within-imputation variation describes the uncertainty of an estimate in a completed data set. Between-imputation variation at a fixed delta describes how the estimates vary across imputations generated under that same delta. Differences between estimates at different delta values instead reflect changes in the sensitivity assumption. They are not ordinary imputation-to-imputation variation.

The target quantity and its scale must be the same across imputations at a given delta. For example, if the target is a treatment-effect coefficient measured in points, every \(Q_j\) must estimate that coefficient in points, and every \(U_j\) must estimate its variance in points squared. A confidence interval under a particular delta describes uncertainty conditional on that sensitivity assumption; it does not include uncertainty about which delta is appropriate.

Rubin’s Rules at a Fixed Delta

For one delta value, first calculate the average of the estimates, the average of their complete-data variance estimates, and the sample variance of the estimates across imputations:

Here \(\bar Q\) is the pooled point estimate, \(\bar U\) is the within-imputation variance, and \(B\) is the between-imputation variance. The total variance estimate and standard error are

The factor \(1+1/m\) includes both the missing-information contribution represented by \(B\) and the finite-\(m\) uncertainty in estimating the pooled result from only \(m\) imputations. When complete-data degrees of freedom are effectively infinite, a commonly used Rubin degrees-of-freedom approximation is

A two-sided approximate \(100(1-\alpha)\%\) confidence interval is \(\bar Q\pm t_{1-\alpha/2,\nu}\sqrt{T}\). This interval uses a \(t\) distribution to reflect the additional uncertainty associated with estimating missing-data variation from a finite number of imputations. If \(B=0\), this displayed degrees-of-freedom expression is not defined; the limiting case has no estimated between-imputation variation, and inference is based on the within-imputation variance, subject to the analysis method’s degrees-of-freedom requirements.

Why the Variance Formula Has These Components

The justification concerns uncertainty about the target parameter, not the conditional variance of a point estimate after a completed data set has been fixed. Write \(\theta\) for the uncertain target parameter, \(O\) for the observed data, and \(D\) for a random completed data set consistent with \(O\). In the usual multiple-imputation uncertainty framework, the law of total variance gives

The first term is uncertainty remaining within a completed data set; \(U_j\) estimates it for imputation \(j\), and \(\bar U\) estimates its average. The second term represents how the target’s conditional expectation varies over possible completed data sets; \(B\), calculated from the \(Q_j\), estimates this component. Thus \(U_j\) is a variance estimate for uncertainty about \(\theta\), not the conditional variance of the fixed point estimate \(Q_j\).

With infinitely many imputations, this decomposition motivates a total variance of approximately \(\bar U+B\). With only \(m\) imputations, the pooled estimate \(\bar Q\) itself has additional simulation uncertainty: its variance includes approximately \(B/m\). Therefore the finite-\(m\) estimate is \(\bar U+B+B/m=\bar U+(1+1/m)B\), as in Rubin’s rules.

Repeat the Calculation Across the Sensitivity Grid

At every delta value, form a separate set of \(m\) estimates and variance estimates, then calculate \(\bar Q(\delta)\), \(\bar U(\delta)\), \(B(\delta)\), \(T(\delta)\), and the corresponding interval. The estimates across delta values form a sensitivity curve, while the interval at each point quantifies imputation and sampling uncertainty under that point’s assumptions.

Assumptions, Diagnostics, and Limits

Rubin’s rules do not make a delta assumption correct. They summarize uncertainty conditional on the chosen scenario. Their usual inferential justification requires proper imputations: the imputation procedure must reflect uncertainty in missing values and relevant model parameters, rather than merely inserting deterministic predictions. At each fixed delta, the imputation and analysis models must be suitable for the variables and estimand, and the completed-data estimate should have a meaningful variance estimate \(U_j\). The imputation process should also have converged sufficiently for the draws to represent the intended distribution.

For a baseline MAR analysis, MAR is an assumption about the missingness mechanism conditional on observed information. A delta-adjusted scenario represents a specified departure from that baseline. At a fixed delta, apply Rubin’s rules to imputations generated under that same adjustment; do not mix imputations from different adjustments to compensate for uncertainty about the missingness mechanism.

Useful checks include confirming that each delta-specific set has the intended number of imputations, that all \(Q_j\) and \(U_j\) refer to the same estimand and scale, and that chains and imputed values are plausible under the imposed adjustment. Examine whether estimates and uncertainty behave sensibly over the grid. A sharp change in the pooled estimate may reflect a meaningful sensitivity to the assumption, while erratic changes within a fixed delta can signal unstable estimation, poor mixing, or too few imputations. Increasing \(m\) can reduce Monte Carlo error in the pooled result, but cannot repair a misspecified imputation or analysis model.

The standard degrees-of-freedom expression above is an approximation, commonly used when complete-data degrees of freedom are effectively infinite. Analyses with limited complete-data degrees of freedom may require a small-sample adjustment appropriate to the analysis. In addition, the interval at each delta does not account for uncertainty in selecting delta, and a grid of scenarios does not itself assign probabilities to those scenarios.

Common Pitfall and Practical Interpretation

A common mistake is to place every estimate from the full sensitivity grid into one Rubin calculation, reasoning that all completed data sets are imputations of the same study. That treats the delta values as if they were random draws from one imputation model. They are instead fixed alternative assumptions chosen for sensitivity analysis. The correction is to apply Rubin’s rules separately at each delta, then compare the resulting pooled estimates and intervals across scenarios.

The comparison across delta values addresses robustness: it shows how conclusions would change under different specified departures from the baseline missingness assumption. It does not create a single pooled estimate averaged over those assumptions. That distinction lets the analysis report both kinds of uncertainty clearly: the interval conditional on each delta scenario, and the variation in conclusions across the sensitivity grid.