Why Estimates Differ Across Imputations
Multiple imputation produces several completed datasets because the missing values are uncertain. An analysis of each completed dataset gives an estimate of the same target quantity, but those estimates need not be identical: different plausible imputations can lead to different results. The spread of these estimates is the between-imputation variation. It is one component of the uncertainty summarized in Rubin’s pooling framework; this tutorial concentrates on how to calculate and interpret that component, not on combining it with the within-imputation variance.
The estimates must be comparable. In particular, each completed dataset must be analyzed using the same estimand, analysis method, scale, and units. If one estimate is a risk difference and another is a log odds ratio, their variation cannot be summarized by a single meaningful between-imputation variance. The imputations should also be generated using a suitable multiple-imputation procedure that reflects uncertainty about missing values and, where relevant, uncertainty in the imputation-model parameters. A very small spread is not evidence by itself that the imputation model is suitable.
Each deviation \(Q_i-\bar Q\) describes how far one completed-data estimate lies from the pooled estimate. Squaring makes negative and positive deviations contribute positively and gives the result units squared. Thus, if \(Q_i\) is measured in millimetres of mercury, \(B\) is measured in squared millimetres of mercury. It is a variance, not a standard deviation; \(\sqrt{B}\) has the original units and can help describe the scale of the estimates’ spread.
The denominator is \(M-1\), not \(M\). The estimates are centered on their own average \(\bar Q\), so their deviations are constrained to sum to zero. That constraint leaves \(M-1\) independent deviations. More formally, under the simplifying assumption that the \(Q_i\) are independent draws with a common mean and variance \(\tau^2\), the sample variance with denominator \(M-1\) has expectation \(\tau^2\). This is why it is the standard estimator of variation across imputations.
Calculating B Step by Step
The calculation is direct, but keeping the centering and denominator explicit prevents common errors. First calculate \(\bar Q\). Then subtract it from each \(Q_i\), square each difference, add the squared differences, and divide by \(M-1\). Keep units attached to the estimates; after squaring, the units of \(B\) are squared as well.
Confirm that all \(M\) estimates target the same scalar quantity and are expressed on the same scale.
Calculate \(\bar Q=\frac{1}{M}\sum_{i=1}^{M}Q_i\).
For each imputation, compute \((Q_i-\bar Q)^2\), then sum these values.
The result is \(B\), an estimated variance across imputations, in squared units of \(Q_i\).
Worked Example: Variation in an Estimated Blood-Pressure Change
Suppose a hypothetical analysis estimates the mean change in systolic blood pressure, in millimetres of mercury, from four completed datasets. The estimates are \(2.1\), \(2.7\), \(1.9\), and \(3.3\) mmHg. Assume they are all estimates of the same mean change, obtained using the same analysis.
First, calculate their average:
The deviations from \(2.5\) mmHg are \(-0.4\), \(0.2\), \(-0.6\), and \(0.8\) mmHg. Their squared deviations sum to:
There are \(M=4\) imputations, so the divisor is \(M-1=3\):
The estimates’ between-imputation variance is \(0.400\text{ mmHg}^2\); its square root is approximately \(0.632\) mmHg. That square root summarizes the scale of spread in the estimates. It is not the standard error of the pooled estimate and is not, by itself, a confidence interval.
Why the Denominator Is M Minus One
The \(M-1\) divisor is not a special adjustment for missingness. It is the usual sample-variance divisor applied to the \(M\) estimates. The short justification below explains the role of centering. Suppose, just for this justification, that \(Q_1,\ldots,Q_M\) are independent and identically distributed, each with mean \(\theta\) and variance \(\tau^2\). Then:
Taking expectations, the first term on the right has expectation \(M\tau^2\). Since the average of \(M\) independent estimates has variance \(\tau^2/M\), the second term has expectation \(M(\tau^2/M)=\tau^2\). Therefore:
This establishes the familiar sample-variance result under the stated independence and common-variance assumptions. In practice, MICE generates imputations through chained conditional models, and estimates from successive draws may be dependent if the chains have not mixed well or if draws are not sufficiently separated. The formula remains the standard calculation of \(B\), but strong dependence can make the observed set of estimates less informative about the underlying between-imputation variation. That is a reason to examine the imputation process, not a reason to replace \(M-1\) with \(M\).
Worked Example: The M Minus One Correction
Consider a hypothetical set of five estimates of a treatment-related difference in symptom-free days: \(10\), \(11\), \(10\), \(13\), and \(12\) days. Their average is:
The deviations are \(-1.2\), \(-0.2\), \(-1.2\), \(1.8\), and \(0.8\) days. Their squared values sum to:
Using the required divisor gives:
Dividing by \(5\) instead would give \(1.36\text{ days}^2\). That is the variance of these five values when treated as the complete set being described, but it is not the standard sample-variance estimate of variation across imputations. The \(M-1\) calculation estimates that variation using the five estimates as a sample.
Interpreting B and Checking Its Behaviour
For a fixed observed dataset, the missing entries are not known. Under an appropriate imputation procedure, different completed datasets represent plausible versions of those entries, and the estimates \(Q_i\) can vary accordingly. \(B\) estimates the variance of the analysis estimate across those imputations. It therefore reflects uncertainty attributable to missing information as represented by the imputation procedure. It is not the variance of the imputed values themselves, nor is it a measure of the full sampling uncertainty in the estimate. The within-imputation variance addresses a different component.
The interpretation depends on \(B\)’s scale and on the estimand. A value of \(0.04\) can be large when \(Q_i\) is a proportion near zero, yet small when \(Q_i\) is a measurement in hundreds of units. Reporting \(B\) without its units and the scale of \(Q_i\) makes its magnitude difficult to understand. The value also depends on the missing-data pattern, the imputation model, and the analysis; it is not a universal property of a dataset.
Worked Example: Between-Imputation Variance for a Risk Difference
In a hypothetical analysis, three completed datasets yield estimated risk differences of \(0.18\), \(0.22\), and \(0.26\), where each risk difference is expressed as a proportion. Assume all three use the same definition of the two groups and the same analysis.
The pooled point estimate is:
The deviations are \(-0.04\), \(0\), and \(0.04\). Thus:
Because the estimates are proportions, \(B\) is in squared proportion units. Its square root is \(0.0400\), or 4 percentage points on the original risk-difference scale. This describes the spread of the three estimates around \(0.22\); it does not say that the risk difference has a 4-percentage-point standard error.
A useful basic diagnostic is to inspect the individual \(Q_i\) values, not just \(B\). A single estimate far from the others can strongly affect the sum of squared deviations. Check whether all estimates came from the same analysis and whether an unusual value corresponds to an unusual completed dataset, a numerical problem, or a genuine consequence of plausible imputed values. For MICE, it is also important that the chains have had adequate time to mix and that the retained imputations are not behaving like repeated copies of nearly the same draw.
A small \(B\) can arise because little information is missing, because the missing values have little influence on this particular estimand, or because the imputations vary too little. These explanations are not interchangeable. Deterministic or overly restrictive imputation can suppress variation, so a small \(B\) should not automatically be read as evidence of little missing-data uncertainty. Conversely, a larger \(B\) is not automatically a defect: it can reflect genuine sensitivity of the estimate to plausible values for missing data.
Common Mistakes and Alternatives
| Approach or error | What it gives or implies | Correction |
|---|---|---|
| Divide by \(M\) | A descriptive variance of these \(M\) values, not the standard sample-variance estimate. | For between-imputation variance, use \(M-1\). |
| Use only the pooled estimate \(\bar Q\) | A point estimate, with no summary of how estimates differ across imputations. | Calculate the squared deviations around \(\bar Q\) to obtain \(B\). |
| Use the within-imputation variances instead | A summary of uncertainty within completed-data analyses, not the spread of the \(Q_i\). | Keep within- and between-imputation variation conceptually separate. |
| Analyze one imputed dataset only | No estimate of between-imputation variation can be calculated. | Use multiple completed datasets; \(B\) is undefined when \(M=1\). |
| Compare estimates with different scales or estimands | A variance with no coherent interpretation. | Ensure every \(Q_i\) targets the same quantity on the same scale. |
The main alternative to multiple imputation is a single completed dataset, whether created by a deterministic method or by one random draw. Such an analysis can produce a point estimate, but it cannot show how that estimate would change across plausible imputations. The earlier material on why single imputation fails explains why treating one completed dataset as if the missing values were known can conceal uncertainty. At the other extreme, calculating the variance with divisor \(M\) answers a descriptive question about the realized set of estimates; it does not replace the standard \(M-1\) estimate used for \(B\).
Finally, \(B\) alone is not a pooled standard error, a confidence interval, a test statistic, or a measure of overall uncertainty. Those require combining the between-imputation component with other information. That combination is a separate step. Here, the essential task is to calculate the variability across \(Q_i\) correctly and interpret it on the right scale.
Check Your Understanding
Work through the questions, then compare your reasoning with the solutions below.
- Four imputations give estimates \(4\), \(5\), \(7\), and \(8\) units. Calculate \(\bar Q\) and \(B\), including the units of \(B\).
- Why is \(M-1\), rather than \(M\), the standard divisor for estimating between-imputation variance?
- Three imputations produce exactly the same estimate. What is \(B\), and what can and cannot be concluded from it?
- Can \(B\) be calculated from a single imputed dataset? Explain.
Solutions. (1) The mean is \(\bar Q=(4+5+7+8)/4=6\) units. The squared deviations are \(4\), \(1\), \(1\), and \(4\), which sum to \(10\text{ units}^2\). Therefore \(B=10/(4-1)=3.33\text{ units}^2\) to three significant figures. (2) Centering the \(M\) estimates on their own mean constrains the deviations to sum to zero. Under independent, equal-variance draws, the expected sum of squared deviations is \((M-1)\tau^2\), so division by \(M-1\) estimates the variance \(\tau^2\). (3) If all three estimates equal \(c\), every deviation from \(\bar Q=c\) is zero, so \(B=0\). This says there is no observed spread among these three estimates; it does not prove that missing-data uncertainty is absent or that the imputation model is adequate. (4) No. With \(M=1\), there are no across-imputation deviations and \(M-1=0\), so \(B\) is undefined.