Why Recompute the Estimand in Each Imputed Dataset?
A delta scenario changes the assumptions used to generate missing values. For each fixed scenario, the result is a set of \(m\) completed datasets, not one completed dataset obtained by averaging the imputations. If the target is a risk ratio, odds ratio, or another nonlinear quantity, calculate that quantity separately in every completed dataset. Then pool the resulting estimates and their variances.
This order matters because nonlinear functions generally do not commute with averaging. For example, suppose two completed datasets give treatment and control risks of 0.20 and 0.10 in the first dataset, and 0.40 and 0.30 in the second. The two risk ratios are \(0.20/0.10=2.00\) and \(0.40/0.30=1.33\), whose arithmetic mean is about 1.67. Averaging the risks first instead gives \(0.30/0.20=1.50\). These are different summaries; the latter does not represent pooling the two dataset-specific risk-ratio estimates.
The delta scenario is held fixed throughout this calculation. The between-imputation variability represents variation among completed datasets under that scenario; it is not a substitute for comparing different delta scenarios. A sensitivity analysis compares results across scenarios after the within-scenario estimates have been computed and pooled.
Pooling on an Appropriate Scale
On the scale chosen for analysis, Rubin’s rules combine the \(m\) estimates \(Q_j\) and their within-imputation variances \(U_j\). The pooled estimate is \(\bar Q\), the average of the estimates. The average within-imputation variance is \(\bar U\), and \(B\) measures the variation among the estimates across imputations. The total variance includes both sources of uncertainty:
The pooled standard error is \(\sqrt{T}\). For a positive ratio, apply these formulas to the log estimates \(q_j=\log(Q_j)\) and their variances \(U_j\) on the log scale. Exponentiating the pooled log estimate gives a pooled ratio. Exponentiating the endpoints of a confidence interval on the log scale gives an interval for the ratio. The interval is generally asymmetric on the ratio scale.
The formulas require valid within-dataset variance estimates, and the usual large-sample pooling approximation works best when the estimates are reasonably well behaved on the chosen scale. The completed datasets must reflect the specified imputation and delta assumptions, and the analysis model and variance method must be suitable for the estimand. For a finite-sample analysis, use an appropriate degrees-of-freedom adjustment; the simple formula below is the conventional large-complete-sample approximation:
A confidence interval on the analysis scale is \(\bar Q \pm t_{df,\,1-\alpha/2}\sqrt{T}\), where \(t_{df,\,1-\alpha/2}\) is the corresponding \(t\) quantile. When pooling log ratios, exponentiate both interval limits after calculating them on the log scale.
Worked Examples
Worked Example: Pooling Two Imputed Risk Ratios
In a hypothetical analysis under one fixed delta scenario, the risk ratios from two completed datasets are 2.00 and 1.50. Their log-scale within-imputation variances are 0.1300 and 0.118889. Calculate the pooled ratio and its approximate 95% confidence interval. Assume the usual large-complete-sample degrees-of-freedom approximation is adequate.
The log estimates are \(Q_1=\log(2.00)=0.693147\) and \(Q_2=\log(1.50)=0.405465\). Thus \(\bar Q=(0.693147+0.405465)/2=0.549306\). The mean within-imputation variance is \(\bar U=(0.1300+0.118889)/2=0.1244445\).
For two imputations, the between-imputation variance is the sum of the two squared deviations from their mean, divided by \(m-1=1\):
Therefore \(T=0.1244445+(1+1/2)(0.0413805)=0.186515\), and the pooled standard error on the log scale is \(\sqrt{0.186515}=0.4319\). The approximate degrees of freedom are \(1[1+0.1244445/(1.5\times0.0413805)]^2\approx9.03\). Using \(t_{9.03,\,0.975}\approx2.26\), the log-scale interval is \(0.549306\pm2.26(0.4319)\), or approximately \((-0.427,\ 1.525)\).
Exponentiating gives a pooled risk ratio of \(\exp(0.549306)=1.732\), with an approximate 95% interval of \((\exp(-0.427),\exp(1.526))\), or \((0.652,\ 4.60)\). Under this scenario, the pooled point estimate is a risk ratio of 1.732. The interval is wide and includes 1, so it does not rule out a risk ratio of 1 at the 5% two-sided level.
Worked Example: A Standardized Risk Ratio
Suppose a hypothetical study estimates risks in two strata, then standardizes both treatment groups to fixed target weights of 0.60 and 0.40. Each stratum has 100 people per treatment group. In completed dataset 1, treatment risks are 0.22 and 0.42, and control risks are 0.10 and 0.30. In completed dataset 2, the corresponding risks are 0.20 and 0.40 for treatment and 0.12 and 0.32 for control. Calculate and pool the standardized risk ratios. Assume independent binomial counts across arms and strata, fixed standardization weights, and use the binomial plug-in variance.
In dataset 1, the standardized treatment risk is \(0.60(0.22)+0.40(0.42)=0.300\), and the standardized control risk is \(0.60(0.10)+0.40(0.30)=0.180\). Thus the risk ratio is \(0.300/0.180=1.667\), and its log is \(Q_1=\log(0.300/0.180)\approx0.510826\).
For a weighted risk with independent strata, the variance is the sum of squared weights times the stratum-specific binomial variances. The treatment risk variance is \(0.60^2(0.22)(0.78)/100+0.40^2(0.42)(0.58)/100=0.00061776+0.00038976=0.00100752\). The control risk variance is \(0.60^2(0.10)(0.90)/100+0.40^2(0.30)(0.70)/100=0.0003240+0.0003360=0.0006600\). Assuming independence between treatment and control, the delta-method variance of the log risk ratio is \(U_1=0.00100752/(0.300)^2+0.0006600/(0.180)^2=0.0111947+0.0203704=0.0315651\).
In dataset 2, the standardized risks are \(0.60(0.20)+0.40(0.40)=0.280\) for treatment and \(0.60(0.12)+0.40(0.32)=0.200\) for control. The risk ratio is \(1.400\), so \(Q_2=\log(1.400)=0.336472\). The treatment risk variance is \(0.60^2(0.20)(0.80)/100+0.40^2(0.40)(0.60)/100=0.000576+0.000384=0.0009600\). The control risk variance is \(0.60^2(0.12)(0.88)/100+0.40^2(0.32)(0.68)/100=0.00038016+0.00034816=0.00072832\). Hence \(U_2=0.0009600/(0.280)^2+0.00072832/(0.200)^2=0.0122449+0.0182080=0.0304529\).
Pooling the log estimates gives \(\bar Q=(0.510826+0.336472)/2=0.423649\), and \(\bar U=(0.0315651+0.0304529)/2=0.0310090\). With \(m=2\), \(B=(0.510826-0.336472)^2/2=0.0151995\). Thus \(T=0.0310090+1.5(0.0151995)=0.0538083\), with standard error \(\sqrt{T}=0.2320\). The pooled standardized risk ratio is \(\exp(0.423649)=1.528\). This estimate compares risks standardized to the stated target stratum weights; it is not the ratio of risks averaged across imputations before the dataset-specific ratios were calculated.
Worked Example: Pooling Odds Ratios
Two hypothetical completed datasets yield treatment and control event counts of 30 and 20 out of 100 in dataset 1, and 25 and 22 out of 100 in dataset 2. Assume independent binomial groups and nonzero counts in all four cells. Calculate the dataset-specific odds ratios and pool them on the log scale.
For a two-by-two table with event and nonevent counts \(a,b\) in treatment and \(c,d\) in control, the odds ratio is \(ad/(bc)\). Its approximate log-scale variance is \(1/a+1/b+1/c+1/d\), under the usual large-sample approximation. In dataset 1, the table counts are \(a=30,\ b=70,\ c=20,\ d=80\), so the odds ratio is \(30(80)/[70(20)]=1.714\), and \(Q_1=\log\{30(80)/[70(20)]\}\approx0.538997\). Its variance is \(U_1=1/30+1/70+1/20+1/80=0.110119\).
In dataset 2, \(a=25,\ b=75,\ c=22,\ d=78\). The odds ratio is \(25(78)/[75(22)]=1.182\), with \(Q_2=\log\{25(78)/[75(22)]\}\approx0.167054\). Its variance is \(U_2=1/25+1/75+1/22+1/78=0.111608\).
The pooled log estimate is \(\bar Q=(0.538997+0.167054)/2=0.353026\), and \(\bar U=(0.110119+0.111608)/2=0.110864\). The between-imputation variance is \(B=(0.538997-0.167054)^2/2\approx0.06917\), not the full squared difference. The total variance is \(T=0.110864+1.5(0.06917)\approx0.21462\), giving a standard error of about \(0.4633\). Exponentiating the pooled log estimate gives an odds ratio of \(\exp(0.353026)=1.423\). This is an odds ratio, not a risk ratio; with common outcomes, the two measures can differ substantially.
Justification, Diagnostics, and Alternatives
The log transformation provides a useful analysis scale for ratios because positive ratios map to the entire real line and multiplicative contrasts become additive. For a risk ratio \(R=p_T/p_C\), its log is \(\log(p_T)-\log(p_C)\). If the two estimated risks are independent, the delta method gives an approximate variance by applying the derivative of the log to each risk:
This follows because the derivatives of \(\log(p_T)-\log(p_C)\) with respect to \(p_T\) and \(p_C\) are \(1/p_T\) and \(-1/p_C\); the variance of a differentiable function is approximated by the variance of its linear expansion. If the risk estimates are dependent, the covariance contribution must also be included. For standardized risks, the risk variances must reflect the standardization procedure and the study design; the simple weighted-binomial calculation above relies on its stated independence and fixed-weight assumptions.
A practical check is to inspect every completed dataset before pooling. Confirm that the intended delta scenario was applied, the analysis uses the same target population and estimand in every dataset, and all required risks and denominators are valid. Then inspect the \(Q_j\), \(U_j\), \(\bar U\), and \(B\) values. A very large \(B\) can indicate that estimates vary sharply across imputations; a very large \(U_j\) can signal sparse data or an unstable variance approximation. Risk ratios and odds ratios may be undefined when a risk denominator is zero, and the usual reciprocal-cell variance for a log odds ratio is not usable when a table cell is zero.
When cells are sparse, consider whether the analysis model and variance estimator are appropriate, and use a method designed for the data structure if the simple large-sample formula is unreliable. Do not silently discard an imputation with a zero cell or replace its estimate with an arbitrary value: that changes the analysis and can distort pooling. More generally, a normal approximation for log estimates can be poor when event counts are small or estimates are highly skewed.
A tempting alternative is to average treatment risks and control risks across datasets and calculate one ratio from those averages. That answers a different question and omits the between-imputation variation of the dataset-specific ratio estimates. Another alternative is to pool risk ratios directly on their original scale. This is not automatically invalid, but it can be unsuitable when the distribution is skewed or the interval could extend below zero even though a ratio must be positive. Pooling log ratios and exponentiating is often more coherent. For standardized or regression-based estimands, calculate the chosen marginal estimand within each completed dataset and use a variance method that accounts for how it was estimated.
Common Pitfalls
- Using the squared difference as \(B\) when \(m=2\). With two estimates, \(B=(Q_1-Q_2)^2/2\); the denominator \(m-1\) applies to the sum of squared deviations from the mean.
- Calculating a ratio only after averaging risks. This does not generally equal pooling the dataset-specific ratios and can conceal their variation across imputations.
- Pooling on one scale and reporting an interval on another without transforming both limits. For a log ratio, calculate the interval on the log scale, then exponentiate each endpoint.
- Using a variance formula without checking its assumptions. Dependence, clustering, standardization, or sparse cells can require a different within-imputation variance estimator.
Check Your Understanding
Answer each question, then compare your work with the solutions.
- Two completed datasets give log risk ratios \(0.20\) and \(0.50\). What is \(B\) for \(m=2\)?
- Why can the ratio of the average risks differ from the average of the dataset-specific risk ratios?
- A pooled log odds ratio is \(0.30\). What is the pooled odds ratio, and on what scale should a symmetric pooling interval first be calculated?
- Name two checks relevant before pooling log odds ratios from completed datasets.
Solutions. 1. \(B=(0.20-0.50)^2/2=0.045\). 2. Division is nonlinear, so in general the ratio of averaged risks does not equal the average of the ratios; it also does not summarize the imputation-to-imputation variability of those ratios. 3. The pooled odds ratio is \(\exp(0.30)=1.350\); calculate the interval on the log odds-ratio scale and exponentiate both endpoints. 4. Check that each table has valid, nonzero cell counts for the usual reciprocal-cell variance formula, and inspect whether the large-sample approximation is plausible given the event counts. Also verify that the same estimand and analysis are used in every completed dataset.