Rubin-Rule Inference After Reference-Based Imputation

From Reference-Based Imputations to an Inferential Result

Reference-based imputation supplies plausible values for missing outcomes under an explicit assumption about post-dropout response. After imputing \(M\) completed data sets, the trial analysis is performed separately in each one. Rubin’s rules combine those analyses so that uncertainty reflects both the usual sampling variability within completed data sets and the variation in estimates across imputations.

The inferential target must be the same in every completed data set. For example, suppose the target is the adjusted difference in mean endpoint change between randomized treatment and reference groups, measured in pain-score points. Each completed-data analysis gives an estimate of that same contrast and an estimated variance for it. Pooling is not a way to remove the assumptions of the reference-based imputation model: it quantifies uncertainty conditional on the imputation and analysis procedures used.

The factor \(1+1/M\) accounts for the fact that the imputation process is estimated from a finite number of imputations. The term \(\bar{U}\) represents the average variance that would remain if the imputed data sets were complete and analyzed separately. The term \(B\) measures how much the treatment-effect estimate changes across imputations; its contribution does not disappear merely because each completed data set has a small standard error.

Degrees of Freedom and Inference

With a large-sample completed-data analysis, the pooled statistic is compared with a \(t\) distribution rather than automatically using a normal distribution. For the convention used here, define the relative between-imputation variance \(r=B/\bar{U}\). When \(M>1\), \(\bar{U}>0\), and \(B>0\), the infinite-complete-data-degrees-of-freedom approximation is

The degrees of freedom \(\nu\) need not be an integer. A two-sided test of \(H_0:\Delta=\Delta_0\) uses \(t=(\bar{Q}-\Delta_0)/SE_{\mathrm{MI}}\) and the \(t\) distribution with \(\nu\) degrees of freedom. A \(100(1-\alpha)\%\) confidence interval is \(\bar{Q}\pm t_{1-\alpha/2,\nu}SE_{\mathrm{MI}}\). These expressions assume that the completed-data estimate is approximately normal with its estimated variance, and that imputations are proper and generated under the specified reference-based procedure. If the completed-data analysis has limited-sample degrees of freedom, an adjustment may be needed; the approximation above treats those degrees of freedom as infinite.

When \(B=0\), the displayed formula for \(\nu\) is undefined because \(r=0\). Under this infinite-complete-data-df approximation, its limiting value as \(B\) approaches zero is infinite. In that boundary case, use the normal distribution for inference. This convention does not mean that the imputation model is known to be correct; it only describes the limiting degrees of freedom implied by this approximation.

Worked Pooling and Inference

Why the Variance Components Matter

The total-variance formula has a short justification. If the imputations were repeated, \(\bar{Q}\) would vary because of sampling uncertainty within a completed data set and because the imputed values—and hence the completed-data estimate—vary across imputations. The within-imputation contribution is estimated by \(\bar{U}\). The finite-\(M\) estimate of the additional variation due to missing values is \((1+1/M)B\). Adding these contributions gives \(T\). This is why averaging the completed-data standard errors, or averaging only the \(U_m\), is not a valid substitute for Rubin’s total variance.

A useful diagnostic is to compare \(B\) with \(\bar{U}\). A large \(B\) relative to \(\bar{U}\) signals substantial imputation-related variability and generally fewer effective degrees of freedom. Also inspect the individual \(Q_m\) values: a result driven by one unusual imputation may indicate unstable imputations or a computational problem. These checks do not establish that the reference-based assumption is scientifically appropriate; they help identify whether the pooling calculation is behaving as expected.

Alternatives, Limitations, and Common Pitfalls

Rubin’s pooling differs from several tempting alternatives. A single imputation treats the filled-in outcomes as if they were observed, omitting uncertainty attributable to imputation. Averaging completed-data estimates but using only \(\bar{U}\) also omits the between-imputation contribution. Conversely, averaging standard errors and then treating that average as the pooled standard error does not implement the variance formula. Rubin’s rules combine estimates and variances, not estimates and standard errors.

• Check scale and units. \(Q_m\) and \(SE_{\mathrm{MI}}\) have the outcome’s units; \(U_m\), \(B\), and \(T\) have squared units. Adding a standard error directly to a variance is a units error.
• Check consistency of the analysis. The same estimand, contrast direction, and analysis method must be used in every imputation. If a contrast is treatment minus reference in one data set and reference minus treatment in another, the pooled result is meaningless.
• Check \(M\) and the boundary. The between-imputation variance requires \(M>1\). If \(B=0\), do not calculate \(1/r\); apply the stated limiting normal convention.
• Keep assumptions in view. Pooling correctly quantifies uncertainty under the imputations supplied. It does not validate the reference-based missing-data assumption or guarantee that the completed-data model is suitable.