Confidence Intervals From Pooled Estimates

Why the Degrees of Freedom Matter for a Pooled Interval

A multiple-imputation confidence interval must reflect two sources of uncertainty: sampling uncertainty within each completed data set and variation among the estimates from different imputations. Rubin’s total variance combines these sources, but that variance alone does not determine the interval’s critical value. The degrees of freedom also matter, particularly when the complete-data analysis has finite degrees of freedom or the number of imputations is modest.

The central task here is to use the pooled point estimate, its total standard error, and an appropriate \(t\) quantile to obtain a two-sided interval. The interval is an approximate multiple-imputation interval: its validity depends on the imputation and analysis being appropriate for the estimand, and on the pooled estimate being adequately approximated by the reference distribution.

For the interval, write \(\nu\) for the degrees of freedom used with the pooled standard error. When the complete-data analysis has finite degrees of freedom, use the Barnard–Rubin adjusted value \(\nu_{\mathrm{BR}}\), rather than automatically using the large-sample Rubin degrees of freedom. The earlier material on Barnard–Rubin small-sample degrees of freedom gives the adjustment; the key point here is that \(\nu\) is the degrees of freedom corresponding to the pooled interval, not the number of observations and not the number of imputations.

Why the Formula Has This Form

The interval follows from an approximate reference distribution for the standardized pooled estimate. Under suitable regularity and imputation conditions, the quantity \((\bar{Q}-Q)/\sqrt{T}\), where \(Q\) is the estimand, is approximated by a \(t\) distribution with the selected degrees of freedom \(\nu\). A central \(1-\alpha\) region of that distribution runs from \(-t_{1-\alpha/2,\nu}\) to \(t_{1-\alpha/2,\nu}\). Thus,

Multiplying the inequalities by the positive quantity \(\sqrt{T}\), then rearranging to put \(Q\) in the middle, gives

This is an approximate repeated-sampling coverage statement under the model and imputation assumptions. It does not mean that, after observing the data, there is a \(1-\alpha\) probability that the fixed estimand lies in this particular interval.

Finite Complete-Data Degrees of Freedom

The complete-data degrees of freedom, \(\nu_{\mathrm{com}}\), describe the reference degrees of freedom that would apply to the corresponding analysis if the data were complete. Their value depends on the analysis and design; it should not be set to \(n-1\) automatically. For example, a regression coefficient’s complete-data degrees of freedom may depend on the fitted model and design, not merely on the sample size.

The Barnard–Rubin adjustment combines the degrees of freedom reflecting missing-information uncertainty with those supported by the complete-data analysis. In the usual scalar setting with positive within- and between-imputation variances, one way to express its components is

The adjusted degrees of freedom are generally no larger than either component: the final expression is a harmonic combination of \(\nu_{\mathrm{old}}\) and \(\nu_{\mathrm{obs}}\). A smaller \(\nu_{\mathrm{BR}}\) gives a larger \(t\) critical value for a fixed confidence level, and therefore a wider interval when the pooled standard error is held fixed. These formulas presuppose that the quantities are well-defined; special cases such as zero or near-zero variance require care and are not resolved by mechanically substituting into a formula.

Calculating an Interval From the Variance Components

When the pooled estimate and standard error are not already reported, construct them from the imputation-specific estimates and variances. The following example shows how the total variance enters before the degrees of freedom are used to choose the critical value.

Assumptions, Diagnostics, and Alternatives

An interval’s arithmetic can be correct while its inferential basis is weak. Before reporting it, check the assumptions that make the pooled estimate, variance, and reference distribution meaningful:

• The estimand and analysis are defined consistently. Each completed-data analysis must estimate the same scalar quantity on the same scale, and each \(U_i\) must estimate the variance of its corresponding \(Q_i\).
• The imputations are suitable for the analysis. The imputation procedure should represent uncertainty about missing values and be compatible with the substantive analysis. Important predictors and relevant structures should not be omitted without justification.
• The pooled approximation is adequate. The \(t\)-based interval is an approximation. Strong skewness, an estimand close to a parameter-space boundary, or unstable variance estimates can make a symmetric interval misleading.
• The complete-data degrees of freedom are appropriate. Use the value appropriate to the complete-data analysis and design. A guessed or mechanically chosen value can distort the \(t\) critical value.
• The simulation is stable enough. Examine trace behavior and compare estimates across imputations. Unstable chains, outlying completed-data estimates, or a very small number of imputations can make pooled summaries noisy. More imputations can reduce Monte Carlo error but do not repair a misspecified imputation model.

The main alternative for a large-sample approximation is a normal critical value, such as \(z_{0.975}\approx1.960\) for a 95% interval. It can be reasonable when the relevant degrees of freedom are very large and the normal approximation is adequate. For finite degrees of freedom, the \(t\) critical value is larger than the corresponding normal value, so substituting 1.960 can produce an interval that is too narrow. Conversely, using \(n-1\) or \(m-1\) as the interval degrees of freedom without justification is not a sound substitute for the appropriate pooled adjustment. A confidence interval on a transformed scale may be preferable when a symmetric interval on the original scale would violate parameter constraints; choosing and pooling transformations is a separate issue from the basic scalar interval here.

Common Mistakes and Practical Interpretation

• Using the wrong quantile. For a two-sided 95% interval, use the 97.5th percentile, \(t_{0.975,\nu}\), not \(t_{0.95,\nu}\). The remaining 5% is split across both tails.
• Using a normal critical value by habit. With small \(\nu_{\mathrm{BR}}\), a normal multiplier can materially understate uncertainty. Select the quantile using the pooled degrees of freedom.
• Confusing the variance and standard error. The interval multiplies \(\sqrt{T}\), not \(T\), by the critical value. Check that \(T\) has squared units and \(SE\) has the estimand’s units.
• Interpreting interval inclusion as a probability about the parameter. A 95% confidence level describes the long-run coverage of the method under its assumptions, not a 95% posterior probability for this fixed parameter after seeing the data.
• Assuming a wide interval means only that the sample was small. Width can also reflect variation among imputations, finite complete-data degrees of freedom, or an imprecise within-imputation analysis.

Report the estimate, interval, confidence level, and units together. For example: “The pooled mean difference was 12.4 symptom-score points (95% confidence interval 6.77 to 18.03 points).” This reports the scale and uncertainty without implying that the interval establishes a causal effect or that every value inside it is equally plausible.