Worked Rubin Pooling For A Logistic Regression Effect

Why the Scale Matters

A logistic regression coefficient for a specified predictor contrast is a log odds ratio. Its exponential is an odds ratio, which is often easier to communicate, but the coefficient and its standard error are on the log-odds scale. Rubin pooling combines estimates and variances on that scale; exponentiation comes afterward. The order matters because exponentiation is nonlinear.

In this tutorial, \(m\) denotes the number of completed data sets. For imputation \(j\), let \(q_j\) be the estimated logistic regression coefficient for the same predictor contrast, and let \(U_j\) be its estimated sampling variance. Thus, if the model reports standard error \(s_j\), then \(U_j=s_j^2\). The target odds ratio is \(\exp(q)\), where \(q\) is the target log-odds-ratio coefficient.

Definition: Rubin pooling for a scalar logistic regression effect first computes the mean coefficient \(\bar{q}\), the mean within-imputation variance \(\bar{U}\), and the between-imputation variance \(B\). The total variance \(T\) combines within- and between-imputation uncertainty. Inference is conducted for \(\bar{q}\) on the log-odds scale; the estimate and confidence limits are then exponentiated to report an odds ratio and its interval.

Assumptions Before Pooling

The \(q_j\) values must estimate the same quantity in every completed data set: the same outcome, predictor contrast, covariate adjustment, coding, and model specification. Each \(U_j\) must estimate the sampling variance of that coefficient in its completed data set. The imputations must be proper for the intended inference, and the imputation and analysis models must support valid estimation of the target under the study’s missing-data assumptions. In particular, the usual justification relies on an adequate imputation model and a defensible missingness assumption, such as missing at random conditional on variables in the imputation model.

The logistic regression fits must also be sufficiently stable for their coefficient estimates and variances to be meaningful. Complete or quasi-complete separation, severe sparsity, a changed contrast in one fit, or an imputation-specific model that fails to converge can make the usual pooled Wald inference unreliable. These issues should be checked before treating the pooled calculation as a solution.

Rubin’s Rules and Finite-Sample Inference

The pooled coefficient is the arithmetic mean of the imputation-specific coefficients. The within-imputation variance describes uncertainty within completed data sets; the between-imputation variance describes how much the estimates vary across them.

\bar{q}=\frac{1}{m}\sum_{j=1}^{m}q_j,\qquad \bar{U}=\frac{1}{m}\sum_{j=1}^{m}U_j,\qquad B=\frac{1}{m-1}\sum_{j=1}^{m}(q_j-\bar{q})^2

The total variance and its standard error are

T=\bar{U}+\left(1+\frac{1}{m}\right)B,\qquad SE(\bar{q})=\sqrt{T}.

The factor \(1+1/m\) accounts for the fact that only a finite number of imputations were used. One way to understand the total-variance formula is to separate uncertainty into two sources: sampling uncertainty within completed data sets, represented by \(\bar{U}\), and uncertainty associated with missing values, represented by \(B\). The latter contributes \(B\), with an additional \(B/m\) adjustment because the pooled estimate uses a finite Monte Carlo set of imputations. Thus the between-imputation contribution is \((1+1/m)B\), not just \(B\).

For a scalar estimate, define the relative increase in variance due to nonresponse as \(r=(1+1/m)B/\bar{U}\). The conventional Rubin degrees of freedom are \(\nu_{\text{old}}=(m-1)(1+1/r)^2\). When the complete-data degrees of freedom \(\nu_{\text{com}}\) are finite, the Barnard–Rubin adjustment also uses

\lambda=\frac{r+2/(\nu_{\text{com}}+3)}{r+1},\qquad \nu_{\text{obs}}=\frac{\nu_{\text{com}}+1}{\nu_{\text{com}}+3}\nu_{\text{com}}(1-\lambda),\qquad \nu_{\text{BR}}=\frac{\nu_{\text{old}}\nu_{\text{obs}}}{\nu_{\text{old}}+\nu_{\text{obs}}}.

Here \(\nu_{\text{obs}}\) is an observed-data degrees-of-freedom component, and \(\nu_{\text{BR}}\) is the adjusted value used for inference. The pooled statistic for testing \(H_0:q=0\) is \(t=\bar{q}/\sqrt{T}\), compared with a \(t\) distribution with \(\nu_{\text{BR}}\) degrees of freedom. A two-sided confidence interval on the log-odds scale is \(\bar{q}\pm t_{0.975,\nu_{\text{BR}}}\sqrt{T}\). Exponentiating the estimate and both endpoints gives the odds ratio and its confidence interval. If complete-data degrees of freedom are effectively infinite, the finite-complete-data adjustment is unnecessary and the conventional Rubin degrees of freedom are used.

Worked Example: Pooling Five Estimates and Reporting an Odds Ratio

Suppose five hypothetical completed data sets yield estimates \(q_j=0.53, 0.55, 0.57, 0.59, 0.61\) for the same binary-exposure contrast in a logistic regression model. Each fit has standard error \(0.216518\). The exposure contrast is exposed versus unexposed, with the other modeled covariates held fixed.

The pooled estimate is \(\bar{q}=(0.53+0.55+0.57+0.59+0.61)/5=2.85/5=0.5700\). The mean within-imputation variance is \(\bar{U}=(0.216518)^2\approx0.0468800\). The deviations from the mean are \(-0.04,-0.02,0,0.02,0.04\), so

B=\frac{0.04^2+0.02^2+0^2+0.02^2+0.04^2}{4} =\frac{0.0040}{4}=0.001000.

Therefore \(T=0.0468800+(1+1/5)(0.001000)=0.0480800\), and \(SE(\bar{q})=\sqrt{0.0480800}=0.2193\). Suppose the finite-complete-data adjustment for this hypothetical analysis gives \(\nu_{\text{old}}=6421.4\) and \(\nu_{\text{obs}}=65.39\). Then \(\nu_{\text{BR}}=(6421.4\times65.39)/(6421.4+65.39)\approx64.73\).

The pooled test statistic is \(0.5700/0.2193\approx2.599\). With \(64.73\) degrees of freedom, the two-sided p-value is approximately \(0.01156\). The 95% log-odds interval, using \(t_{0.975,64.73}\approx1.997\), is

0.5700\pm1.997(0.2193) =[0.1321,\ 1.008].

The pooled odds ratio is \(\exp(0.5700)\approx1.768\). Exponentiating the interval endpoints gives \([\exp(0.1321),\exp(1.008)]\approx[1.141,2.740]\). In this hypothetical model, the estimated odds for the exposed group are about 1.77 times those for the unexposed group at the specified covariate values. The interval describes uncertainty in that odds-ratio estimate under the stated modeling and imputation assumptions; it is not a probability statement about the fixed parameter.

Worked Example: Using the Adjusted Degrees of Freedom for a Borderline Result

In a second hypothetical analysis, the pooled log-odds-ratio estimate is \(\bar{q}=0.2100\), and the total variance is \(T=(0.1061)^2=0.01125721\). The degrees-of-freedom calculations give \(\nu_{\text{old}}=176.3\) and \(\nu_{\text{obs}}=87.62\). Combining them gives

\nu_{\text{BR}}=\frac{176.3(87.62)}{176.3+87.62} =\frac{15447.406}{263.92}\approx58.53.

The standard error is \(0.1061\), and the test statistic is \(0.2100/0.1061\approx1.979\). Using a \(t\) reference distribution with \(58.53\) degrees of freedom gives a two-sided p-value of approximately \(0.0525\). At \(\alpha=0.05\), this result does not cross the conventional significance threshold.

For a 95% interval, \(t_{0.975,58.53}\) is approximately \(2.001\). Thus the log-odds interval is \(0.2100\pm2.001(0.1061)\), or approximately \([-0.0023,0.4223]\). On the odds-ratio scale, the estimate is \(\exp(0.2100)\approx1.234\), and the interval is \([\exp(-0.0023),\exp(0.4223)]\approx[0.9977,1.526]\). The interval includes a log odds ratio of zero, equivalently an odds ratio of one, consistent with the two-sided p-value being slightly above 0.05.

A Second Full Pooling Calculation

Worked Example: Pooling Four Imputation-Specific Fits

Four hypothetical logistic regression fits for the same contrast give coefficient estimates \(0.30, 0.40, 0.50, 0.60\), with standard errors \(0.20, 0.22, 0.18, 0.24\). The pooled estimate is \(\bar{q}=(0.30+0.40+0.50+0.60)/4=0.4500\). Squaring the four standard errors gives \(0.0400,0.0484,0.0324,0.0576\), so \(\bar{U}=(0.1784)/4=0.04460\).

The deviations from \(0.4500\) are \(-0.15,-0.05,0.05,0.15\). Hence \(B=(0.0225+0.0025+0.0025+0.0225)/3=0.01667\), and

T=0.04460+\left(1+\frac{1}{4}\right)(0.01667) =0.06543,\qquad SE(\bar{q})=\sqrt{0.06543}\approx0.2558.

The relative increase is \(r=0.02083/0.04460\approx0.4671\). Suppose the complete-data degrees of freedom are \(80\). Then \(\lambda=(0.4671+2/83)/(1+0.4671)\approx0.3348\), and \(\nu_{\text{obs}}=(81/83)(80)(1-0.3348)\approx51.93\). Also, \(\nu_{\text{old}}=3(1+1/0.4671)^2\approx29.59\), giving \(\nu_{\text{BR}}=(29.59)(51.93)/(29.59+51.93)\approx18.85\).

The test statistic is \(0.4500/0.2558\approx1.759\), with a two-sided \(t\)-based p-value of about \(0.095\). Using \(t_{0.975,18.85}\approx2.096\), the 95% log-odds interval is \(0.4500\pm2.096(0.2558)\approx[-0.0862,0.9862]\). The odds ratio is \(\exp(0.4500)\approx1.568\), with interval \([\exp(-0.0862),\exp(0.9862)]\approx[0.9174,2.681]\). The interval includes one, matching the lack of evidence against \(H_0:q=0\) at the 0.05 level.

Checks, Alternatives, and Common Pitfalls

Before reporting results, verify that the coefficient represents the same contrast in every imputation and that all fits converged. Compare the imputation-specific estimates and standard errors: unusually large variation, extreme coefficients, or a single very large standard error may signal sparse data, separation, a coding mismatch, or unstable fitting. A large \(B\) relative to \(\bar{U}\) indicates that the imputations contribute appreciable uncertainty; it is not by itself evidence of an error. The number of imputations and Monte Carlo stability also matter for reliable pooling.

Rubin pooling is not the same as averaging odds ratios. The mean of \(\exp(q_j)\) is generally not \(\exp(\bar{q})\), and it does not supply the Rubin pooled variance. Nor is an average of imputation-specific p-values a pooled test. Complete-case analysis uses less information and can target a biased result under some missingness patterns; single imputation generally fails to carry imputation uncertainty through the analysis. A normal-reference interval may be a reasonable large-sample approximation, but when finite degrees of freedom are available, the adjusted \(t\) reference distribution is the relevant choice for this calculation.

Wrong: “The pooled odds ratio is the average of the odds ratios from the completed data sets.” Correction: pool the log-odds-ratio coefficients and their variances, then exponentiate the pooled coefficient.
Wrong: “The 95% interval means there is a 95% probability that the fixed odds ratio is inside these endpoints.” Correction: under repeated sampling and the model assumptions, the interval procedure has 95% coverage; the computed interval is not a probability distribution for the fixed parameter.
Wrong: “A p-value of 0.0525 is the probability that the null hypothesis is true.” Correction: it is the probability, under the null and the reference model, of a test statistic at least as extreme as the observed one.

These calculations rely on a meaningful common coefficient, adequate variance estimates, and a suitable imputation and analysis procedure. They do not repair model misspecification, nonconvergence, or an indefensible missingness assumption. Report the odds ratio with its confidence interval, and retain the pooled log-odds coefficient, standard error, and degrees of freedom in the analysis record so the inference can be checked.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

A pooled coefficient is \(0.40\). What is the corresponding odds ratio, approximately?
Why is the log-odds coefficient pooled before exponentiation, rather than averaging the imputation-specific odds ratios?
A 95% log-odds interval is \([-0.10,0.50]\). Give the odds-ratio interval approximately and state whether it includes the null value.
A pooled test has \(t=2.1\) and \(\nu_{\text{BR}}=12\). Should its finite-sample p-value be calculated using a standard normal or a \(t\) reference distribution?

Solutions. (1) \(\exp(0.40)\approx1.492\). (2) Rubin’s rules combine the coefficient and its variance on the analysis scale; exponentiation is nonlinear, so averaging exponentiated coefficients does not give the pooled estimate or its uncertainty. (3) Exponentiation gives \([\exp(-0.10),\exp(0.50)]\approx[0.9048,1.649]\); it includes one, the null odds ratio. (4) Use a \(t\) reference distribution with 12 degrees of freedom, because the stated finite-sample pooled degrees of freedom are not infinite.