Keeping The Estimand And Scale Consistent Across Imputations

Why Consistency Comes Before Pooling

Rubin’s rules combine estimates of one quantity across completed data sets. They do not make estimates comparable merely because they came from the same model command or have similar-looking names. A coefficient for one reference category, a coefficient for another contrast, and an effect expressed in different units may all be valid within their own analyses while representing different quantities. Averaging them does not produce a meaningful pooled estimate.

The practical task is therefore to define the target first, then calculate an estimate and its variance for that target in every imputation. This is especially important for contrasts, categorical predictors, standardized effects, and nonlinear transformations. Earlier material on linear contrasts, marginal effects, and transformed estimates gives tools for carrying out those calculations; here the emphasis is on ensuring the calculations refer to the same target throughout.

For a scalar target, Rubin’s pooled estimate is \(\bar{q}=m^{-1}\sum_{j=1}^{m}\hat{q}_j\). The average within-imputation variance is \(\bar{U}=m^{-1}\sum_{j=1}^{m}U_j\), and the between-imputation variance is \(B=(m-1)^{-1}\sum_{j=1}^{m}(\hat{q}_j-\bar{q})^2\). The total variance is \(T=\bar{U}+(1+1/m)B\). These expressions presuppose that every \(\hat{q}_j\) estimates the same \(q\) and every \(U_j\) is a variance for that same \(q\). A numerically computable \(B\) is not evidence that this presupposition holds.

What Must Match Across Imputations?

At minimum, check the meaning of the comparison, the coding or contrast used to express it, the units, and the scale of the reported quantity. The fitted model form and target population should also be comparable. A different coefficient name can be harmless if it is converted to the same contrast; identical coefficient names can be misleading if the coding or interpretation changed.

• Contrasts: specify which groups or predictor values are compared and in what direction.
• Reference categories: verify the coefficient is a comparison with the intended reference, or construct the intended contrast from the fitted coefficients.
• Units: express slopes or differences in one common unit before pooling, and transform the variances accordingly.
• Transformations: decide whether the target is on the model scale or a transformed scale, such as log odds or odds ratio. Obtain an estimate and variance for the chosen target in every imputation.
• Standardization: state which standard deviation or other scaling quantity defines the target. Different scaling conventions can define different effects.

A common linear transformation is straightforward. If \(\hat{q}_j^*=c\hat{q}_j\), then the corresponding variance is \(U_j^*=c^2U_j\). More generally, for a vector of coefficients \(\hat{\boldsymbol{\beta}}_j\), a common linear contrast \(L^\mathsf{T}\hat{\boldsymbol{\beta}}_j\) has estimated variance \(L^\mathsf{T}V_jL\), where \(V_j\) is that coefficient vector’s estimated covariance matrix. The contrast and covariance calculation must be done within each imputation before pooling.

For a nonlinear function \(f\), the target might instead be \(f(\theta)\). One can calculate \(\hat{q}_j=f(\hat{\theta}_j)\) in every imputation and estimate its variance on that scale, for example by the delta method when appropriate. Averaging \(f(\hat{\theta}_j)\) and then applying \(f\) to an average are generally different operations. Which one is relevant depends on the estimand, not on which computation is more convenient.

Units and Transformations Need Their Own Variance

Changing units is a re-expression of the same target only when the conversion is applied consistently. For a slope, changing the predictor unit changes the numerical coefficient; it also changes its variance. If a coefficient per one unit is \(\hat{\beta}\), the coefficient per ten units is \(10\hat{\beta}\), and its variance is \(100\) times the original variance. Pooling ten-unit coefficients with one-unit coefficients, or converting only the estimates but not their variances, does not apply Rubin’s rules to a common quantity.

Nonlinear transformations require a more deliberate choice. If the target is a log odds ratio, keep estimates and variances on the log scale for pooling, then transform the pooled estimate for presentation if desired. If the target is an odds ratio itself, each imputation must provide an odds-ratio estimate and a variance on a suitable scale for pooling. In general, the average of transformed estimates is not the transformation of the average estimate. This is not a computational nuisance: the two operations can correspond to different targets.

Scale Is Part of the Target

A probability difference and a log odds ratio are not interchangeable merely because both describe a fitted model. Likewise, a model coefficient and an average predicted contrast are different quantities in general. When the target is a predicted probability difference, calculate that difference in each completed data set, along with its variance, and pool those differences. Do not substitute a pooled regression coefficient unless the target and model make that coefficient exactly the desired quantity.

A Short Justification and Practical Diagnostics

A common linear re-expression preserves the pooling structure if both estimates and variances are transformed. For a scalar multiplier \(c\), let \(\hat{q}_j^*=c\hat{q}_j\) and \(U_j^*=c^2U_j\). Then \(\bar{q}^*=c\bar{q}\), \(B^*=c^2B\), and \(\bar{U}^*=c^2\bar{U}\). Consequently,

Thus the standard error changes by \(|c|\), as it should under a change of units. This justification does not say that unlike targets can be made equivalent by multiplying them by a convenient constant. The conversion must genuinely map each imputation’s estimate to the same defined quantity.

Before pooling, use a short audit:

Useful warning signs include a coefficient’s sign reversing when the reference group changes, implausible magnitude changes associated with unit differences, and a large increase in between-imputation variance after estimates are supposedly aligned. Also check software defaults: factor-level ordering, reference categories, omitted levels, and prediction grids can differ across model fits. Identical variable labels are not a substitute for checking these details.

Common Mistake and Limits

Wrong reasoning: “The software returned one coefficient per imputation with the same name, so I can pool them. If the units differ, Rubin’s rules will account for that through \(B\).” This is incorrect. \(B\) describes variation among estimates of a common target; it does not harmonize distinct contrasts, units, or scales. A unit mismatch can distort both the pooled estimate and the variance, while a reference-category mismatch can change the comparison itself.

Correction: define the target and re-express every estimate and its variance on that target’s scale before pooling. If the model outputs do not contain enough information to construct the same contrast—for example, the needed coefficients or their covariance are unavailable—refit or extract the required quantities rather than pooling incomplete outputs. If aligned estimates still differ substantially, examine model specification, data handling, numerical instability, and the imputation-specific results. Correct harmonization cannot eliminate genuine uncertainty or repair an inappropriate model.

Compared with pooling a single regression coefficient, pooling a derived contrast or prediction can better match the scientific question but requires calculating its uncertainty within each imputation. Transforming a pooled estimate is convenient when the target is naturally defined on the model scale and the transformation is for presentation; it is not generally equivalent to pooling estimates of a transformed target. Separate analyses of each imputation without pooling avoid the alignment calculation only by forfeiting the intended combined inference. In every case, validity also depends on appropriate models and missing-data assumptions; consistent labels and scales alone do not guarantee valid inference.