Diagnosing Implausible Copy-Reference Distributions

Why Diagnose a Copy-Reference Distribution?

A copy-reference (CR) imputation for a participant who drops out of the active arm uses the reference arm’s outcome model to predict that participant’s missing outcomes. In a sequential model, the prediction can depend on outcomes already observed for that participant. This conditioning is useful: it lets the predicted trajectory reflect the participant’s recorded history while using reference-arm relationships to project forward.

That combination can also produce predictions that deserve scrutiny. A fitted reference covariance matrix may be unstable, the conditional mean may be far outside a clinically credible range, or the participant’s observed history may have little support in the reference arm. A prediction can be mathematically well-defined yet rely on extrapolation. Diagnostics therefore ask not only whether software produced a number, but also whether the number is supported by the fitted model and the observed reference data.

Definition: Let $H$ denote the vector of outcomes observed up to dropout and $F$ the vector of missing future outcomes. Under a multivariate normal reference-arm model, write $$ \begin{pmatrix}H\\F\end{pmatrix} \sim N\left[ \begin{pmatrix}\mu_{R,H}\\\mu_{R,F}\end{pmatrix}, \begin{pmatrix} \Sigma_{R,HH}&\Sigma_{R,HF}\\ \Sigma_{R,FH}&\Sigma_{R,FF} \end{pmatrix} \right]. $$ A CR prediction conditions this reference-arm model on the participant’s actual observed history $H=h$. Its conditional mean is $$ \mu_{R,F}+\Sigma_{R,FH}\Sigma_{R,HH}^{-1}(h-\mu_{R,H}), $$ and its conditional covariance is $$ \Sigma_{R,FF}-\Sigma_{R,FH}\Sigma_{R,HH}^{-1}\Sigma_{R,HF}. $$

The model uses reference-arm parameters, but the history $h$ can come from a participant assigned to the active arm. The conditional prediction therefore carries the participant’s observed history forward through the reference model’s estimated associations. These formulas presume that the relevant reference-arm covariance matrix is estimable and positive definite, that the multivariate normal model is an adequate working model, and that the observed history is not so far outside the reference-arm data that the conditional calculation amounts to unsupported extrapolation.

What Can Make a Prediction Implausible?

Three warning signs are especially important. First, covariance estimation may be poor: a small reference arm, sparse follow-up, or strong collinearity can make the fitted covariance unstable. Second, the conditional mean can be extreme because the observed history is far from the reference-arm mean or because the fitted conditional regression is sensitive to small changes in that history. Third, a history may be poorly represented in the reference arm. Even if the equations return a finite conditional mean and a positive conditional variance, the reference data may offer little evidence for that prediction.

These issues are related but not interchangeable. A poorly estimated covariance can make predictions unstable across model fits. An extreme conditional mean can instead be a direct consequence of a genuinely unusual history. A mismatch in histories is a question about support: the reference arm may not contain participants with comparable observed trajectories. Diagnostics should examine all three rather than treating a plausible-looking mean as proof that the full predictive distribution is sound.

Worked Example: An Extreme Conditional Mean From an Unusual History

Suppose a hypothetical continuous outcome is measured in points. In the reference arm, the history $H$ has mean 10 and variance 4; the next outcome $F$ has mean 12 and variance 9; and their covariance is 3. A participant assigned to the active arm drops out after an observed history value of 20 points. Calculate the CR conditional distribution.

For a bivariate normal model, the conditional mean and variance are $$ E(F\mid H=h)=\mu_{R,F}+\frac{\operatorname{Cov}(F,H)}{\operatorname{Var}(H)}(h-\mu_{R,H}), \qquad \operatorname{Var}(F\mid H)=\operatorname{Var}(F)-\frac{\operatorname{Cov}(F,H)^2}{\operatorname{Var}(H)}. $$ Substituting the hypothetical values gives $$ E(F\mid H=20)=12+\frac{3}{4}(20-10)=12+7.5=19.5\text{ points}, $$ and $$ \operatorname{Var}(F\mid H)=9-\frac{3^2}{4}=9-2.25=6.75\text{ points}^2. $$ Thus the conditional standard deviation is $\sqrt{6.75}=2.598$ points.

The conditional mean, 19.5 points, is far above the reference-arm marginal mean of 12 points. The observed history is also 5 reference-arm standard deviations above its mean, since $(20-10)/\sqrt{4}=5$. This is a warning that the prediction depends on an unusual history, not proof that 19.5 is mathematically impossible. Investigators should check whether that history is credible, whether it is represented by any reference-arm participants, and whether the reference-model relationship is reliable in that region.

Covariance Fit and Conditional Stability

The inverse $\Sigma_{R,HH}^{-1}$ and the cross-covariance $\Sigma_{R,FH}$ determine how the observed history changes the conditional mean. If $\Sigma_{R,HH}$ is nearly singular, small changes in observed values or in the estimated covariance can produce large changes in this adjustment. A fitted covariance matrix should be symmetric and positive definite for a non-degenerate multivariate normal model. A non-positive conditional variance, or a covariance matrix that fails these checks, signals a numerical or fitting problem rather than a usable predictive distribution.

Positive definiteness alone is not enough. A matrix can be technically positive definite but nearly singular, so the inverse is sensitive to small perturbations. Inspect eigenvalues or a condition number, check convergence and warnings from the fitting procedure, and examine whether estimates change substantially under reasonable refits or resampling. For multiple-imputation procedures with parameter draws, inspect predictions across draws as well: a few highly influential covariance draws can create a wide or irregular predictive distribution. These checks do not replace the specified imputation model; they reveal when its estimated ingredients may be unreliable.

Worked Example: A Near-Singular Covariance Can Amplify a History Contrast

Consider a hypothetical reference model with two observed history components and one future outcome, all measured on the same scale. Let $$ \Sigma_{R,HH}= \begin{pmatrix}1&0.999\\0.999&1\end{pmatrix}, \qquad \Sigma_{R,FH}=\begin{pmatrix}0.02&-0.02\end{pmatrix}, \qquad \Sigma_{R,FF}=1. $$ The eigenvalues of $\Sigma_{R,HH}$ are 1.999 and 0.001, so the history covariance is positive definite but nearly singular.

The conditional regression coefficient is $$ \Sigma_{R,FH}\Sigma_{R,HH}^{-1}=(20,-20). $$ For a history residual $h-\mu_{R,H}=(0.1,-0.1)^\mathsf{T}$, the adjustment to the reference marginal mean is $$ (20,-20) \begin{pmatrix}0.1\\-0.1\end{pmatrix}=2+2=4. $$ Thus the predicted conditional mean is 4 outcome units above the reference marginal mean. The conditional variance remains positive: $$ 1-\Sigma_{R,FH}\Sigma_{R,HH}^{-1}\Sigma_{R,HF} =1-0.8=0.2. $$

This calculation is internally valid, but the large coefficients show that the prediction is sensitive to a contrast between two highly correlated history components. The residual vector is also unusual in the direction with very little reference-arm variation. A diagnostic review should examine the eigenvalue, the fitted covariance’s stability, and whether such opposing history values are supported by the reference data. The example illustrates why checking only for a positive conditional variance can miss a practically unstable prediction.

Checking Whether Histories Match the Reference Arm

For a participant with history $h$, a useful multivariate measure of distance from the reference-arm history distribution is the squared Mahalanobis distance, $$ D^2=(h-\mu_{R,H})^\mathsf{T}\Sigma_{R,HH}^{-1}(h-\mu_{R,H}). $$ It accounts for different scales and correlations among history measures. A large value flags a history that is unusual under the fitted reference model. It is a diagnostic, not a universal pass/fail cutoff: its interpretation depends on the number of history components, the sample size, the model fit, and the intended use of the imputations. Also compare observed histories directly with the reference-arm data, using plots or summaries by dropout pattern where appropriate. A distance calculated from an uncertain covariance estimate should itself be treated cautiously.

Worked Example: Detecting Limited Reference-Arm Support

Suppose a hypothetical trial records two history outcomes. In the reference arm, their mean vector is $(10,20)$, and their covariance matrix is diagonal with variances 4 and 9. A participant has history $(16,29)$. The standardized deviations are $(16-10)/2=3$ and $(29-20)/3=3$, so $$ D^2=\frac{(16-10)^2}{4}+\frac{(29-20)^2}{9}=9+9=18. $$ The distance is $\sqrt{18}=4.243$ in standardized multivariate units.

For illustration, let the reference-arm future outcome have mean 30, variance 4, and covariances 1 with each history component. The conditional mean is $$ 30+\begin{pmatrix}1/4&1/9\end{pmatrix} \begin{pmatrix}6\\9\end{pmatrix} =30+1.5+1=32.5. $$ The covariance specification is internally valid because the conditional variance is $4-1/4-1/9=3.639$ outcome units squared, which is positive. Nevertheless, the history is 3 standard deviations above the reference mean on both components. The value 32.5 is therefore a model-based extrapolation for this history; validity of the algebra does not establish strong empirical support for it.

A Diagnostic Workflow and Its Limits

Check the fitted covariance.
Confirm symmetry and positive definiteness; inspect small eigenvalues, convergence warnings, and sensitivity to reasonable refits. A non-positive conditional variance is not a distribution to use.

Inspect predicted means and spreads.
Summarize conditional means and standard deviations by dropout time and pattern. Compare them with outcome ranges and clinically meaningful units; look for extreme values, unexpectedly tiny spreads, or unusually large spreads.

Assess history support.
Compare active-arm histories at dropout with reference-arm histories, including their multivariate distances and plots. Identify predictions driven by regions with few or no comparable reference participants.

Check predictive draws, not only means.
Review simulated missing outcomes and parameter draws for implausible tails, boundary violations, or strong sensitivity to a few fitted parameter values. A reasonable conditional mean can coexist with an unreasonable predictive distribution.

Document the implication.
State which predictions are supported by observed reference histories and which rely on extrapolation or unstable covariance estimates. A warning should prompt investigation and transparent reporting, not an unreported change to the assumed model.

CR differs from a marginal reference prediction because CR conditions on the participant’s observed history; the marginal prediction does not use that history in the same conditional way. The earlier material on J2R distinguishes another reference-based assumption, but neither a different reference rule nor a marginal prediction automatically resolves poor support or covariance instability. The appropriate comparison depends on the estimand and assumptions. These diagnostics do not determine which missing-data assumption is scientifically correct, and a good fit to observed reference-arm data cannot verify the unobserved outcomes after dropout.

A common mistake is to reason, “The covariance matrix is positive definite and the software returned a finite mean, so the prediction is acceptable.” Positive definiteness establishes a mathematical condition for the fitted normal distribution; it does not establish stable estimation, plausible tails, or reference-arm support for the participant’s history. The correction is to check matrix conditioning, conditional predictions, and history overlap together, then report limitations that remain. Do not silently trim or replace extreme imputations: doing so changes the procedure and needs a justified, explicitly described model.

Key takeaway: A copy-reference prediction is conditional on both the fitted reference-arm model and the participant’s observed history. Diagnose the covariance fit, the resulting conditional distribution, and the history’s support in the reference arm; no single finite prediction or matrix check answers all three questions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In one sentence, explain why a CR conditional mean may differ from the reference-arm marginal mean.
A fitted covariance matrix is positive definite but has a very small eigenvalue. What concern does this raise, and what should be checked?
Why does a large Mahalanobis distance flag a potential problem without proving that an imputation is invalid?
Can a positive conditional variance establish that a prediction is empirically supported? Explain.

Solutions. (1) The conditional mean includes an adjustment based on the participant’s observed history and the reference model’s history-to-future associations. (2) The matrix may be nearly singular, making its inverse and conditional predictions sensitive to small changes; examine convergence, eigenvalues, prediction sensitivity, and reference-arm support. (3) The distance flags an unusual history under the fitted model, but it is not a universal cutoff and depends on model fit and available data. (4) No. A positive variance establishes a mathematical property of the fitted conditional distribution, not adequate estimation, plausible values, or comparable reference-arm histories.