Common Error: Replacing Only Post-Dropout Means

Why post-dropout means are not enough

Reference-based imputation rules describe how outcomes after a participant drops out are predicted using a reference-arm distribution or a related reference-based assumption. A common error is to state only the means assigned to those unobserved outcomes. Means describe the centers of predictions, but they do not say how variable the outcomes are or how outcomes at different visits move together. Those features matter whenever the analysis uses more than one missing outcome per participant.

The distinction is important even when two proposed imputations agree on every post-dropout mean. They can produce different distributions of individual trajectories, different conditional predictions given observed history, and different uncertainty for a treatment contrast. A reference-based predictive distribution is not fully defined until its covariance assumptions are also stated.

The calculations below use a multivariate normal model to make the issue explicit. This is a modeling assumption, not a mathematical requirement for every reference-based method. In a different distributional model, the same principle applies: a location or mean alone does not determine the full predictive distribution.

Conditional predictions require covariance information

Let \(Y_H\) denote the vector of outcomes observed before or at dropout, and \(Y_M\) the vector of outcomes missing after dropout. Conditional on a specified parameter value, suppose their joint distribution is multivariate normal:

Here, \(\Sigma_{HH}\) describes variation in observed outcomes, \(\Sigma_{MM}\) describes variation and association among missing outcomes, and \(\Sigma_{MH}\) describes their covariance with the observed history. The covariance matrix must be symmetric and positive semidefinite; for the conditional formula below, \(\Sigma_{HH}\) must also be invertible.

In reference-based imputation, a rule such as jump to reference (J2R) or copy-reference concerns how a reference-based distribution is used for post-dropout predictions. The chosen post-dropout means are only part of that specification. The model must also state which covariance structure is used, including the association between observed history and future outcomes and the association among future outcomes. The phrase “use the reference means” does not, by itself, determine whether those covariance features come from the reference arm, are retained from the treatment arm, or follow some other stated model.

Deriving the conditional distribution

The conditional formula follows from a standard property of a jointly multivariate normal vector. To see why the covariance appears, define \(A=\Sigma_{MH}\Sigma_{HH}^{-1}\) and form the residual vector \(R=Y_M-\mu_M-A(Y_H-\mu_H)\). Its covariance with \(Y_H\) is zero:

Because \(R\) and \(Y_H\) are jointly normal, zero covariance implies independence. Thus conditioning on \(Y_H=h\) leaves the distribution of \(R\) unchanged, while \(Y_M=\mu_M+A(h-\mu_H)+R\). Its conditional mean is therefore \(\mu_M+A(h-\mu_H)\). Expanding the residual covariance gives:

The last equality uses the symmetry of the covariance matrix and the definition of \(A\). This derivation shows why replacing a mean while leaving the covariance unspecified cannot produce a unique conditional predictive distribution.

Covariance choices affect complete predictive distributions

A mean-only replacement can conceal more than one modeling decision. For example, a procedure might set post-dropout means to reference-arm values but retain treatment-arm covariance estimates. Another might use both reference-arm means and reference-arm covariance estimates. A third might impose independence among missing visits. These are not equivalent predictive models, even if they share the same post-dropout mean vector.

For multiple missing visits, the off-diagonal entries of \(\Sigma_{MM}\) determine how outcomes co-vary. The cross-block \(\Sigma_{MH}\) determines how observed history informs future predictions. If a reference-based rule is intended to use a reference-arm predictive distribution, the analyst should state which of these covariance components are taken from that distribution and how any remaining components are specified. Replacing only a block of means does not answer those questions.

Assumptions, diagnostics, and failure modes

The formulas and examples rely on a specified multivariate normal model conditional on its parameters. In practice, the analyst must also have enough information to estimate or justify the covariance parameters for the relevant arm and dropout pattern. If the observed-history covariance block is singular, the displayed inverse does not exist; a different model or a justified regularization strategy is needed rather than silently applying the formula. Estimated covariance matrices should be checked for symmetry and positive semidefiniteness, since an invalid matrix cannot define a proper normal predictive distribution.

Diagnostics should assess the predictive distribution, not just whether the imputed averages look plausible. Useful checks include:

• Compare the assumed visit-specific variances and correlations with estimates from the relevant observed data, while recognizing that post-dropout covariance is not directly observed for participants who have dropped out.
• Inspect simulated trajectories across visits. Check whether the covariance assumptions create implausibly smooth, erratic, or tightly linked changes.
• For each dropout pattern, verify that the covariance dimensions match the observed and missing outcome vectors and that the resulting conditional covariance is valid.
• Compare predictions conditional on different observed histories. A model that ignores cross-covariance may fail to reflect history even where such dependence is intended.
• Track the variability of analysis summaries across imputations. Identical imputed means do not imply identical between-imputation or within-imputation uncertainty.

A further issue is parameter uncertainty. The formulas give a predictive distribution conditional on fixed parameter values. In reference-based multiple imputation, parameter draws and outcome draws contribute to the final uncertainty, as covered in the earlier material on parameter draws for reference-based multiple imputation and Rubin-rule inference after reference-based imputation. A mean-only description also leaves unclear what covariance parameters are estimated or drawn. Omitting covariance uncertainty can understate uncertainty, while an unstable covariance estimate can generate erratic predictions.

Common mistake and practical comparison

Wrong reasoning: “The reference-based rule says to use a post-dropout mean of 5 points, so the imputation distribution is specified.” This gives a center but does not state the spread, the association across missing visits, or how observed history relates to the missing outcomes. Even after assuming normality, infinitely many covariance matrices can accompany that mean vector.

Correction: State the distributional family and its covariance assumptions along with the mean rule. For a multivariate normal predictive model, specify the covariance blocks needed to calculate both \(\mu_{M\mid H}\) and \(\Sigma_{M\mid H}\). Then check that the resulting covariance is valid and assess whether the implied trajectories and uncertainty are reasonable for the intended reference-based assumption.

The main alternatives differ in what they specify, not merely in their labels. A fully specified reference-based predictive model uses a stated reference-based mean and covariance structure. Retaining a treatment-arm covariance while changing post-dropout means is a different, hybrid assumption and should be reported as such. Imposing independence among missing outcomes is another explicit covariance assumption, not a consequence of changing means. Each choice can be defensible for a stated purpose, but none should be presented as if it follows automatically from the mean rule.