Why Uncertainty Must Follow the Derived Quantity
A derived quantity may be deterministic once its source variables are known, but it is not known when those sources are missing. For example, a total score is fixed by its item values, yet different plausible imputations of missing items can produce different totals. If an analysis uses those totals, its result can differ across completed datasets as well. Recalculating the derived quantity in each completed dataset is how this variation enters the analysis.
This is a distinct issue from merely preserving an algebraic identity. Passive imputation defines the derived variable from its sources in each dataset. Here the focus is what follows for estimates and their uncertainty: calculate the analysis quantity separately in each completed dataset, then combine those estimates and their variance estimates using multiple-imputation rules. Averaging source variables first and deriving once can instead hide variation and may target a different quantity.
The \(Q_l\) values must estimate the same scientific quantity on the same scale. For each completed dataset, recalculate any passive term from its inputs, fit the specified analysis, and extract the estimate and its variance. \(W\) describes sampling uncertainty within completed datasets; \(B\) describes how estimates vary across imputations. The factor \(1+1/m\) accounts for the finite number of imputations when estimating total uncertainty.
Assumptions and a Variance Justification
Rubin’s variance rules are appropriate when the imputations are proper draws that reflect uncertainty about missing values and relevant model parameters, the analysis and imputation are sufficiently congenial for the target, and the completed-data estimates are approximately normal with usable within-dataset variance estimates. These conditions are approximations in practice. They are less reliable with very few imputations, highly skewed or boundary-limited estimates, severe model misspecification, or an analysis model that is incompatible with the imputation procedure. Recomputing a derived term correctly does not by itself guarantee these conditions.
A useful justification comes from the variance decomposition. Let \(Q\) denote the estimate that would be obtained if the missing values were known, and let \(D\) denote a possible completed dataset. Conditional on the observed data, total uncertainty can be separated into uncertainty within a completed dataset and variation in the conditional estimate across possible completions:
In a finite set of imputations, \(W\) estimates the first component and \(B\) estimates variation in the second component. The observed between-imputation spread is itself estimated from only \(m\) draws; the multiplier \(1+1/m\) supplies the finite-imputation adjustment in Rubin’s total variance formula. Thus, omitting \(B\) discards uncertainty attributable to not knowing the missing values, while reporting \(B\) alone discards ordinary sampling uncertainty.
Retain each imputed dataset separately; do not replace the collection with one averaged dataset.
Apply its defining formula to the appropriate source values in each completed dataset.
Estimate the same target \(Q_l\) and its within-dataset variance \(U_l\) in every completed dataset.
Calculate \(\bar{Q}\), \(W\), \(B\), and \(T\); then assess whether the pooled approximation and model assumptions are plausible.
Worked Examples
Worked Example: A Total Score Across Imputations
Suppose a hypothetical assessment has four items, and some item values are missing. In each completed dataset, the total score is recalculated as the sum of the four items. An analysis estimates the mean total score. For three completed datasets, suppose the estimates are \(Q_1=12\), \(Q_2=14\), and \(Q_3=13\) points, with within-dataset standard errors \(2\), \(2\), and \(2.5\) points. Thus \(U_1=4\), \(U_2=4\), and \(U_3=6.25\) points squared.
The pooled estimate is \( \bar{Q}=(12+14+13)/3=13 \) points. The within-imputation variance is \( W=(4+4+6.25)/3=14.25/3=4.75 \) points squared.
The deviations from the pooled estimate are \(-1\), \(1\), and \(0\) points, so \( B=\{(-1)^2+1^2+0^2\}/(3-1)=2/2=1 \) point squared. Therefore, \( T=4.75+(1+1/3)(1)=4.75+1.3333=6.0833 \) points squared, and the pooled standard error is \( \sqrt{6.0833}=2.466 \) points.
The score total is recomputed from the imputed item values in every dataset, so differences in plausible item completions contribute to the spread of the mean-score estimates and hence to \(B\). The pooled estimate is 13 points, with a pooled standard error of about 2.466 points. The standard error is not an additional score value; it quantifies uncertainty in the estimated mean.
Worked Example: Pooling a Ratio Rather Than Dividing Pooled Components
Consider a hypothetical analysis in which the target in each completed dataset is a ratio of two estimated quantities, on a common scale. Suppose the three dataset-specific numerator and denominator estimates are \((10,20)\), \((20,25)\), and \((30,40)\). Assume each dataset’s ratio estimate has a within-dataset standard error of \(0.05\). These values are illustrative; the analysis must use a variance estimator appropriate to the actual ratio and its data.
Calculate the ratio in each completed dataset: \(Q_1=10/20=0.5000\), \(Q_2=20/25=0.8000\), and \(Q_3=30/40=0.7500\). The pooled estimate of this target is \( \bar{Q}=(0.5000+0.8000+0.7500)/3=2.0500/3=0.6833 \). The within-imputation variance is \(W=(0.05^2+0.05^2+0.05^2)/3=0.002500\).
The deviations from \(0.6833\), using the exact mean \(41/60\), are \(-11/60\), \(7/60\), and \(4/60\). Therefore, \( B=\{(121+49+16)/3600\}/2=186/7200=0.02583 \). The total variance is \( T=0.002500+(1+1/3)(0.02583)=0.002500+0.03444=0.03694 \), so the pooled standard error is \( \sqrt{0.03694}=0.1922 \).
Dividing the average numerator by the average denominator would give \( (10+20+30)/(20+25+40)=60/85=0.7059 \), not \(0.6833\). These are different summaries: the first averages the dataset-specific ratios, while the second is the ratio of the pooled component estimates. The intended estimand determines which is appropriate. If the target is a ratio calculated in each completed dataset, pool those ratios and their variances. If the target is instead a ratio of pooled components, its uncertainty must account for the joint uncertainty and covariance of those components; simply averaging them and treating them as known is not sufficient.
A ratio can be skewed or constrained by its domain, so a normal approximation on the raw ratio scale may be poor. The numerical pooling illustrates the calculation, not a guarantee that a normal-theory interval is suitable. Diagnostics and, when needed, an appropriate transformation or alternative inferential method should be considered.
Worked Example: Propagating Covariance Into a Product Estimate
Suppose an analysis reports a product \(Q=ab\) of two estimated quantities, perhaps a contrast constructed from two model components. In each completed dataset, estimate \(a\), \(b\), their variances, and their covariance. For a product, the delta method gives the approximate within-dataset variance \( U \approx b^2\operatorname{Var}(a)+a^2\operatorname{Var}(b)+2ab\operatorname{Cov}(a,b) \). This follows from the gradient \((b,a)\) of \(g(a,b)=ab\): the approximate variance is the gradient multiplied by the covariance matrix and its transpose.
In three hypothetical completed datasets, suppose the values \((a,b)\) are \((4.0,3.0)\), \((4.2,3.1)\), and \((3.8,2.9)\). In all three, let \(\operatorname{Var}(a)=0.25\) and \(\operatorname{Var}(b)=0.16\); let the respective covariances be \(0.10\), \(0.08\), and \(0.09\). The product estimates are \(Q_1=4.0(3.0)=12.00\), \(Q_2=4.2(3.1)=13.02\), and \(Q_3=3.8(2.9)=11.02\), in product units.
For dataset 1, \(U_1=3^2(0.25)+4^2(0.16)+2(4)(3)(0.10)=2.25+2.56+2.40=7.21\). For dataset 2, \(U_2=3.1^2(0.25)+4.2^2(0.16)+2(4.2)(3.1)(0.08)=2.4025+2.8224+2.0832=7.3081\). For dataset 3, \(U_3=2.9^2(0.25)+3.8^2(0.16)+2(3.8)(2.9)(0.09)=2.1025+2.3104+1.9836=6.3965\).
The pooled estimate is \( \bar{Q}=(12.00+13.02+11.02)/3=12.0133 \) product units. The within-imputation variance is \(W=(7.21+7.3081+6.3965)/3=6.9715\). The deviations from the pooled estimate are approximately \(-0.01333\), \(1.0067\), and \(-0.9933\), giving \(B\approx1.0001\). Thus \( T=6.9715+(1+1/3)(1.0001)=8.3050 \), and the pooled standard error is approximately \( \sqrt{8.3050}=2.882 \) product units.
This calculation carries both covariance within each completed dataset and variation across completed datasets into the pooled uncertainty. Omitting the covariance term can materially change \(U_l\); treating \(a\) and \(b\) as independent without justification is not generally valid.
Diagnostics, Alternatives, and Limits
A practical diagnostic is to inspect the completed-data estimates \(Q_l\), their standard errors, and the resulting \(B\) relative to \(W\). Large differences in \(Q_l\) may reflect substantial uncertainty from missing values, but can also reveal unstable imputation models, sparse categories, extreme imputations, or a derived formula that is sensitive near a boundary. Check that every derived quantity was recalculated from its sources in every dataset, that all datasets use the same units and coding, and that \(U_l\) corresponds to the same estimand as \(Q_l\). A derived value that is constant across imputations despite incomplete source values deserves particular scrutiny.
The main alternative shortcut is to average imputed source values first and then calculate the derived quantity once. For nonlinear functions, that generally changes the target because \(g\{E(X)\}\) need not equal \(E\{g(X)\}\). It also fails to represent the spread in derived values across plausible imputations. A second shortcut is to calculate estimates in each dataset but pool only the point estimates, reporting no uncertainty; this omits both within-dataset sampling variation and the between-imputation contribution. By contrast, calculating the target and its variance in every dataset and applying Rubin’s rules preserves both components under the stated assumptions.
These steps do not repair an unsuitable imputation model or guarantee that the imputation and substantive analysis are compatible. They also do not make a poor normal approximation reliable for a highly skewed derived estimate. With few imputations, \(B\) can be noisy; increasing \(m\) can reduce Monte Carlo error in pooled quantities, but cannot correct bias from model misspecification. For estimates with awkward distributions, inference on a suitable transformed scale or a method designed for that estimand may be preferable, with the target and interpretation stated clearly.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- In three completed datasets, an estimate is \(5\), \(7\), and \(6\), with within-dataset variances \(1\), \(1\), and \(1\). Find \(\bar{Q}\), \(W\), \(B\), and \(T\).
- Why can calculating a nonlinear derived quantity from averaged imputed inputs differ from averaging the derived quantity calculated in each completed dataset?
- For \(Q=ab\), why can the within-dataset variance not generally be calculated by adding only the separate variance contributions from \(a\) and \(b\)?
- A derived quantity has the same value in every completed dataset even though one of its source variables was imputed. Name one check to make before trusting the result.
Solutions. (1) \( \bar{Q}=(5+7+6)/3=6\), \(W=1\), and \(B=\{(-1)^2+1^2+0^2\}/2=1\). Thus \(T=1+(1+1/3)(1)=2.333\). (2) In general, a nonlinear function does not commute with averaging: \(g\{E(X)\}\) need not equal \(E\{g(X)\}\), and the first calculation can suppress variation across imputations. (3) The two estimates may be correlated; the delta-method variance includes \(2ab\operatorname{Cov}(a,b)\), which is zero only under an appropriate zero-covariance condition. (4) Verify that the derived variable was recalculated from the imputed source in every completed dataset and that the formula, inputs, and update order were implemented correctly.