Reference-Based Multiple Imputation Methods

Introduction

Missing outcome data are common in longitudinal clinical trials. Patients may discontinue treatment because of adverse events, lack of efficacy, withdrawal of consent, administrative reasons, or other circumstances.

When outcomes are missing after discontinuation, investigators must make an assumption about what those patients would have experienced had their outcomes continued to be observed.

A standard multiple-imputation analysis may assume that the observed data provide an appropriate basis for predicting the missing outcomes. But in a randomized clinical trial, investigators may have additional information available: the observed trajectory of patients in the reference treatment arm.

Reference-based multiple imputation uses this information explicitly.

What Is Reference-Based Multiple Imputation?

Reference-based multiple imputation is a class of missing-data sensitivity analysis methods in which the distribution of missing outcomes for patients in one treatment arm is informed by the observed outcomes of patients in another arm.

The reference arm therefore supplies a clinically interpretable trajectory against which post-discontinuation outcomes can be modeled.

For example, consider a randomized two-arm trial:

Suppose an experimental-treatment patient discontinues at Week 12.

A reference-based assumption may specify that after Week 12, the patient's unobserved outcomes should evolve according to the trajectory associated with the reference arm.

Different reference-based methods define this borrowing differently.

Why Is This Useful?

The primary analysis of a clinical trial may use a missing-at-random assumption or another model-based approach.

However, the plausibility of that assumption can be difficult to establish from observed data alone.

Reference-based methods provide a way to examine a scientifically meaningful alternative scenario.

For example:

• What if patients who discontinue active treatment subsequently behave like control patients?
• What if they retain their accumulated treatment benefit but subsequently follow the control trajectory?
• What if they immediately revert to the reference treatment trajectory?
• How sensitive is the treatment effect to assumptions about post-discontinuation outcomes?

The Missing-Data Setup

Suppose a patient has repeated measurements:

where \(T\) is the number of scheduled visits.

Suppose the patient is observed through visit \(k\) but then discontinues.

are observed, while:

are missing.

The goal of multiple imputation is to generate plausible values for those missing observations while preserving uncertainty about what the unobserved values could have been.

Observed and Missing Components

It is useful to partition the outcome vector into observed and missing pieces:

Under ordinary multiple imputation, one models the conditional distribution:

Reference-based imputation modifies the assumptions governing this distribution for patients who discontinue treatment.

The Role of the Reference Arm

Let \(A_i\) denote treatment assignment.

Suppose:

denotes experimental treatment and:

denotes reference treatment.

Reference-based methods use the observed reference-arm data to inform the distribution of missing outcomes for selected experimental-arm patients.

The exact way that information is borrowed defines the particular reference-based method.

The Three Major Reference-Based Strategies

Three commonly discussed approaches are:

The terminology and exact implementation can vary somewhat across formulations, so the statistical analysis plan should define the chosen assumption precisely.

Jump to Reference

Jump to Reference, commonly abbreviated J2R, is one of the most widely discussed reference-based assumptions.

The central idea is: once an experimental-treatment patient discontinues, their future outcomes behave as though they had moved immediately to the reference trajectory.

Suppose an experimental patient has a favorable trajectory through Week 12 and then discontinues.

Under J2R, the post-discontinuation outcomes do not continue to benefit from the experimental treatment trajectory.

Clinical Interpretation of J2R

J2R can be interpreted as a conservative scenario in settings where treatment benefit is assumed to disappear after treatment discontinuation.

It is especially intuitive when treatment discontinuation means that the patient no longer receives the pharmacologic intervention responsible for the benefit.

However, "conservative" is not a universal statistical property.

Copy Increments in Reference

Copy Increments in Reference, or CIR, makes a different assumption.

The patient retains their observed position at discontinuation, but subsequent changes are assumed to follow the reference-arm increments.

Suppose the outcome at discontinuation is:

and the reference arm typically changes by \(-2\) units per visit.

Under a simplified CIR illustration, future values might follow:

The patient therefore retains their achieved level at dropout but follows the reference trajectory thereafter.

J2R Versus CIR

The distinction is easiest to understand graphically.

Thus, CIR can preserve more of the patient's accumulated treatment effect than J2R.

Copy Reference

Copy Reference, or CR, is another reference-based strategy.

Conceptually, the missing outcomes are generated to follow the reference-arm distribution rather than continuing the experimental-arm trajectory.

The exact mathematical formulation depends on the implementation.

In practical work, analysts should not rely solely on the acronym. The SAP should explicitly state:

• Which reference distribution is used
• How the patient's observed history enters the imputation model
• How treatment-arm covariance is handled
• What happens at the discontinuation visit
• Whether the reference trajectory is copied absolutely or through increments

Reference-Based Methods Are Not Single Imputations

A common misunderstanding is that reference-based imputation assigns one deterministic future trajectory to every patient.

That is not the purpose of multiple imputation.

Instead, the method generates multiple plausible values from an assumed conditional distribution.

For imputation \(m\):

where \(f_{\mathrm{RB}}\) denotes the reference-based conditional distribution.

Each imputed dataset therefore represents one plausible realization of the missing data under the chosen assumption.

Why Multiple Imputation Is Necessary

Suppose 100 patients have missing outcomes.

There is uncertainty about each patient's missing value.

If we filled in one value for each patient and analyzed the completed dataset as though those values were known, the resulting standard errors would generally be too small because they would ignore imputation uncertainty.

Multiple imputation addresses this by creating \(M\) completed datasets.

Each dataset contains different plausible draws for the missing observations.

The General Multiple-Imputation Workflow

A Longitudinal Example

Consider a randomized clinical trial with measurements at baseline and Weeks 4, 8, 12, 16, and 20.

Suppose an experimental patient has the following observed trajectory:

The reference arm provides observed trajectories from which post-discontinuation behavior can be learned.

What J2R Is Doing in This Example

Under J2R, the patient's observed experimental history through Week 12 remains part of the conditioning information.

But the future distribution is shifted toward the reference-arm behavior after discontinuation.

Conceptually:

is constructed using reference-arm information rather than simply extrapolating the experimental-arm treatment effect indefinitely.

Why the Covariance Structure Matters

Reference-based MI is not merely about replacing one mean with another.

Longitudinal observations are correlated.

For example:

and:

A realistic imputation model therefore needs to account for the covariance structure of repeated measurements.

A multivariate normal model is one common framework.

Multivariate Normal Framework

Let the repeated outcomes follow:

where:

• \(\mu\) is the vector of visit-specific means.
• \(\Sigma\) is the within-patient covariance matrix.
• \(T\) is the number of repeated measurements.

Partition the mean vector and covariance matrix into observed and missing components:

and:

The Conditional Normal Distribution

Under the multivariate normal model, the conditional distribution of the missing observations given the observed observations is:

This equation is fundamental to many model-based multiple-imputation procedures.

Reference-based methods modify the assumptions governing the relevant mean trajectory for the missing portion.

Mean Trajectory Under J2R

Conceptually, suppose the experimental treatment has produced a treatment difference through the time of discontinuation.

Under J2R, that treatment difference is not assumed to continue indefinitely after discontinuation.

The post-discontinuation mean is instead constructed from the reference trajectory.

A simplified representation is:

with the patient's observed history still contributing to the conditional distribution.

This distinction is important. J2R does not mean that every missing value is simply replaced by the reference-arm mean.

The imputation remains conditional on the patient's observed history and the specified covariance structure.

J2R as a Sensitivity Scenario

Suppose the primary analysis assumes missing at random.

The investigator might then specify J2R as a sensitivity scenario:

If the treatment conclusion remains stable across clinically plausible assumptions, confidence in the robustness of the conclusion increases.

Reference-Based MI and the Estimand

Missing-data methodology cannot be separated from the estimand.

Consider a treatment-policy estimand.

The question may be: What would the treatment effect be regardless of treatment discontinuation?

Alternatively, an estimand may be constructed around a hypothetical scenario in which treatment discontinuation is handled differently.

Reference-based assumptions therefore need to be consistent with the scientific question being answered.

Reference-Based MI Does Not "Fix" Missing Data

There is no statistical procedure that can recover unobserved outcomes with certainty.

Reference-based MI replaces an unidentified missing-data problem with an explicit modeling assumption.

The resulting analysis answers: What would the treatment effect look like under this particular post-discontinuation scenario?

That is fundamentally different from proving what actually happened after patients stopped providing data.

J2R, CIR, and MAR

These approaches should be regarded as different assumptions about the unobserved data, not as competing algorithms where one is universally correct.

Reference-Based MI Versus LOCF

Reference-based MI is fundamentally different from Last Observation Carried Forward.

LOCF might replace:

with:

for both future visits.

This creates a deterministic trajectory:

Reference-based MI instead generates a distribution of plausible future outcomes.

Reference-Based MI Versus Simple Mean Imputation

Simple mean imputation might assign:

where \(\bar{Y}_{R}\) is a reference-arm mean.

This discards important information about:

• The patient's observed trajectory
• Within-patient correlation
• Visit-specific behavior
• Covariates
• Imputation uncertainty

Reference-based MI is substantially richer because it models a conditional distribution rather than inserting a single fixed value.

A Numerical Illustration

Suppose an experimental patient has an observed outcome of 60 at the time of discontinuation.

Suppose the reference arm has an average future change of \(-3\) units per visit.

A simplified CIR scenario might produce an expected trajectory:

A simplified J2R scenario might instead place the patient's future trajectory directly on the reference distribution, conditional on their observed history.

The important distinction is that these are probability distributions, not deterministic sequences.

Multiple Imputed Datasets

Suppose we generate \(M=50\) imputed datasets.

Each dataset contains different imputed values:

We then perform the same prespecified analysis on each dataset.

Suppose the treatment effect estimates are:

The pooled treatment effect is the average:

Rubin's Rules

Let \(U_m\) denote the estimated variance from imputed dataset \(m\).

The within-imputation variance is:

The between-imputation variance is:

Rubin's total variance is:

The square root of \(T\) gives the pooled standard error:

Fraction of Missing Information

A useful diagnostic is the fraction of missing information.

A simplified large-sample expression is:

A larger value indicates that a greater proportion of the uncertainty in the estimate is associated with missing-data uncertainty.

How Many Imputations Should Be Used?

The number of imputations should be sufficient to make Monte Carlo error from the imputation process negligible for the intended analysis.

Older examples frequently use a small number such as 5 or 10 imputations.

Modern clinical-trial analyses often use substantially more, particularly when the fraction of missing information is high or precise tail-area inference is important.

Values such as \(M=50\), \(M=100\), or more may be reasonable depending on the analysis.

Reference-Based Imputation in a Randomized Trial

Suppose there are two treatment groups:

A typical workflow is:

Pattern-Mixture Perspective

Reference-based MI is closely connected to the pattern-mixture framework for missing data.

A pattern-mixture model decomposes the joint distribution according to the missingness pattern:

where \(R\) represents the missingness pattern.

For example, one pattern might correspond to patients who remain observed through Week 20, while another corresponds to patients who discontinue after Week 12.

The distribution of future outcomes for the latter pattern is not fully identified by the observed data.

Reference-based assumptions provide a scientifically interpretable way to complete that specification.

Why the Missing-Data Assumption Is Not Identified

Suppose an experimental patient discontinues at Week 12.

Their Week 16 outcome is unobserved.

No statistical analysis of the observed dataset can tell us with certainty what the patient's Week 16 outcome actually was.

Different assumptions can produce different plausible distributions:

Both can be compatible with the observed data.

This is why sensitivity analysis is so important.

Reference-Based MI as an Explicit Sensitivity Analysis

A robust clinical-trial missing-data strategy may therefore include a collection of assumptions.

The goal is not necessarily to find one "correct" imputation.

The goal is to determine whether the substantive conclusion is robust to reasonable assumptions about the missing data.

Combining Reference-Based MI With Delta Adjustment

Reference-based methods can also be considered alongside delta-adjusted sensitivity analyses.

Suppose the imputed missing outcome under a reference-based method is \(Y_{\mathrm{mis}}^{RB}\).

A delta adjustment can define:

where \(\delta\) is prespecified.

For example, one could examine a sequence of increasingly unfavorable post-discontinuation assumptions.

R Implementation: Conceptual Setup

A complete production implementation requires careful handling of the covariance matrix, missingness patterns, treatment assignment, and the precise reference mapping.

The following R code illustrates the structure of a simplified multivariate normal imputation workflow.

library(MASS)

M <- 100

observed <- c(52, 47, 43, 40)

mu_reference <- c(50, 48, 46, 45, 44, 43)

Sigma <- matrix(
  c(
    9,  5,  4,  3,  2,  2,
    5,  9,  5,  4,  3,  2,
    4,  5,  9,  5,  4,  3,
    3,  4,  5,  9,  5,  4,
    2,  3,  4,  5,  9,  5,
    2,  2,  3,  4,  5,  9
  ),
  nrow = 6,
  byrow = TRUE
)

The covariance matrix describes the assumed longitudinal dependence among visits.

Conditional Normal Imputation

For a generic multivariate normal model, the conditional mean can be calculated as:

obs_idx <- 1:4
mis_idx <- 5:6

mu_obs <- mu_reference[obs_idx]
mu_mis <- mu_reference[mis_idx]

Sigma_oo <- Sigma[obs_idx, obs_idx]
Sigma_mo <- Sigma[mis_idx, obs_idx]
Sigma_om <- Sigma[obs_idx, mis_idx]
Sigma_mm <- Sigma[mis_idx, mis_idx]

conditional_mean <- mu_mis +
  Sigma_mo %*%
  solve(Sigma_oo) %*%
  (observed - mu_obs)

conditional_cov <- Sigma_mm -
  Sigma_mo %*%
  solve(Sigma_oo) %*%
  Sigma_om

The resulting conditional distribution is:

A random imputation can then be generated using:

imputed <- MASS::mvrnorm(
  n = 1,
  mu = as.vector(conditional_mean),
  Sigma = conditional_cov
)

Creating Multiple Imputed Datasets

imputed_datasets <- vector("list", M)

for (m in seq_len(M)) {

  imputed <- MASS::mvrnorm(
    n = 1,
    mu = as.vector(conditional_mean),
    Sigma = conditional_cov
  )

  complete_data <- c(
    observed,
    imputed
  )

  imputed_datasets[[m]] <- complete_data
}

Each element of imputed_datasets represents one completed dataset.

Why Parameter Uncertainty Matters

The covariance matrix and mean parameters are themselves estimated from the observed data.

If we treat those estimated parameters as known, we may understate uncertainty.

A full multiple-imputation procedure therefore needs to account for parameter uncertainty as well as residual variation.

Conceptually, the imputation process involves:

Reference-Based MI and MMRM

Reference-based multiple imputation is frequently discussed alongside longitudinal models such as the mixed model for repeated measures, or MMRM.

An MMRM may model:

where \(j\) indexes visit and \(X_i\) represents baseline covariates.

The residual covariance matrix captures correlation among repeated measurements.

Reference-based MI can use the fitted longitudinal structure as part of the machinery for generating missing outcomes under a specified post-discontinuation scenario.

Why MMRM and Reference-Based MI Are Complementary

An MMRM addresses the observed longitudinal data structure.

Reference-based MI specifies how missing post-discontinuation outcomes are handled under the sensitivity scenario.

A Simplified MMRM Example in R

library(nlme)

fit <- gls(
  outcome ~ baseline + treatment * visit,
  correlation = corSymm(
    form = ~ visit | subject
  ),
  weights = varIdent(
    form = ~ 1 | visit
  ),
  data = observed_data,
  na.action = na.omit
)

summary(fit)

The exact model used in a clinical trial should be prespecified and may include baseline covariates, treatment-by-visit interactions, appropriate covariance structures, and other protocol-defined terms.

Monotone Missingness

Reference-based MI is particularly intuitive when dropout produces a monotone missing-data pattern.

For example:

Once the patient discontinues, all subsequent measurements are missing.

However, real clinical datasets can contain intermittent missing values.

For example:

The imputation strategy must therefore specify how intermittent missingness is handled separately from permanent post-discontinuation dropout.

Intermittent Missingness vs. Treatment Discontinuation

These are not necessarily the same missing-data mechanism.

Reference-based assumptions are particularly focused on the post-discontinuation portion of the missing data.

Reference-Based MI and Treatment Discontinuation

The analysis should define exactly when the reference-based assumption begins.

Suppose a patient discontinues treatment immediately after the Week 12 visit.

One approach is to regard Week 12 as the final observed treatment outcome and apply the reference-based assumption beginning with Week 16.

The analysis specification should make this timing explicit.

Covariates in Reference-Based MI

Baseline covariates can be important predictors of future outcomes.

Examples include:

• Baseline disease severity
• Age
• Sex
• Prior treatment
• Baseline laboratory measurements
• Stratification factors

When appropriate, these variables can be included in the imputation model to improve predictive accuracy and preserve important design features.

Preserving Randomization

Treatment assignment should generally remain fixed throughout the imputation process.

Reference-based MI should not change the randomized treatment assignment.

Instead, reference-arm information is used to define the assumed distribution of missing outcomes for the relevant patients.

Handling Treatment Arms With More Than One Reference

Some studies contain more complex designs:

• Two experimental doses and one placebo arm
• Active comparator plus placebo
• Multiple reference treatments
• Combination-treatment factorial designs

In these situations, the SAP should specify which treatment provides the reference trajectory for each missing-data pattern.

The reference choice should have a scientific rationale.

Reference-Based MI in Multi-Arm Trials

The choice should never be made solely because one reference group produces a more favorable or unfavorable result.

Common Mistakes

1. Calling reference-based MI an observed-data method. The post-discontinuation assumption concerns outcomes that are not observed.
2. Assuming J2R means replacing every missing value with the control mean. J2R is a conditional distributional assumption, not simple mean imputation.
3. Ignoring covariance. Longitudinal outcomes are correlated, and the covariance structure matters.
4. Using a single imputed dataset. That discards important imputation uncertainty.
5. Ignoring parameter uncertainty. Estimated model parameters should not automatically be treated as known.
6. Calling J2R universally conservative. The direction of conservatism depends on the endpoint and clinical context.
7. Failing to define the discontinuation point. The timing of the transition to the reference-based assumption must be explicit.
8. Changing the reference method after seeing results. Sensitivity assumptions should be prespecified.
9. Choosing a reference arm because it gives the desired result. The reference should be selected based on scientific justification.
10. Ignoring the estimand. The imputation assumption must correspond to the treatment-effect question.

Reference-Based MI vs. Delta Adjustment

These two approaches are often discussed together.

Reference-Based MI and Tipping-Point Analysis

A tipping-point analysis asks how much the missing-data assumptions would need to change before the substantive conclusion changes.

For example, suppose a treatment effect remains statistically significant under J2R.

An analyst might then investigate increasingly unfavorable assumptions until the confidence interval includes the null value.

This can help quantify the robustness of the conclusion.

Reference-based MI can therefore be part of a broader sensitivity-analysis framework.

What Does a Robust Result Look Like?

Suppose the primary analysis produces:

with a 95% confidence interval of:

Suppose J2R produces:

with:

and CIR produces:

with:

The treatment conclusion appears relatively stable across these scenarios.

By contrast, if J2R changes the conclusion substantially, that indicates that the inference is sensitive to assumptions about post-discontinuation outcomes.

What Should Be Reported?

A clinical-trial report should make the sensitivity analysis reproducible.

Useful reporting elements include:

• The estimand
• The primary missing-data assumption
• The reference-based method
• The definition of treatment discontinuation
• The reference treatment arm
• The longitudinal model
• The covariance structure
• The number of imputations
• The imputation algorithm
• How parameter uncertainty was handled
• The analysis model applied after imputation
• The pooling method
• Results across sensitivity scenarios

Example Statistical Analysis Plan Language

A protocol or SAP might specify a sensitivity analysis along the following conceptual lines:

The actual SAP should provide substantially more implementation detail than this conceptual description.

Diagnostics After Imputation

Reference-based MI should be checked carefully.

Useful diagnostics include:

• Distribution of imputed outcomes
• Observed versus imputed means by visit
• Observed versus imputed standard deviations
• Reference-arm trajectory comparison
• Distribution of individual imputed trajectories
• Number and proportion of imputed observations
• Convergence or numerical diagnostics
• Consistency across random seeds

Plotting Observed and Imputed Trajectories

A useful diagnostic plot distinguishes observed from imputed measurements.

For example, an individual trajectory might show:

This can reveal obvious implementation errors, such as imputed values that are completely disconnected from the observed history or implausibly narrow imputation distributions.

Checking the Imputed Distribution

Suppose the observed Week 12 outcome has a mean of 40.

If the reference-based method generates a Week 16 distribution with mean 41 and very small variance, the result deserves investigation.

The analyst should ask:

• Was the reference trajectory correctly specified?
• Was the covariance matrix correctly indexed?
• Were baseline covariates handled correctly?
• Was the correct discontinuation visit used?
• Was the conditional distribution calculated correctly?

Number of Imputations and Monte Carlo Error

Reference-based MI introduces simulation variability.

If too few imputations are used, the pooled estimate can fluctuate slightly between runs.

Increasing \(M\) reduces this Monte Carlo error.

This does not mean that infinitely many imputations are required. The number should be large enough that simulation error is negligible for the intended inference.

Reproducibility

A production implementation should preserve:

• Program version
• Software version
• Random-number seed
• Input dataset version
• Parameter estimates
• Covariance estimates
• Imputation count
• Reference-based transformation
• Analysis dataset derivation

This is particularly important when reference-based MI is used in a regulated clinical-trial environment.

Validation Strategy

The imputation program should be independently checked.

Useful validation approaches include:

• Independent programming
• Known-value test cases
• Simulation studies
• Extreme-case testing
• Missingness-pattern testing
• Comparison with an established implementation
• Reproducibility testing across seeds

Extreme-Case Testing

Extreme-case tests are particularly useful.

For example, create a test patient whose observed history exactly matches the reference trajectory.

Under a properly implemented reference-based method, the resulting imputation should behave consistently with the specified reference distribution.

Another useful test is to create a patient with a very strong experimental trajectory and verify that the chosen method does not accidentally continue the experimental treatment effect when the specified assumption says that it should transition toward the reference trajectory.

Reference-Based MI With Binary Outcomes

Reference-based MI is not limited to continuous outcomes.

For binary outcomes, the imputation model must respect the binary nature of the endpoint.

For example:

A logistic model or another appropriate categorical-data model may be used.

The same conceptual principle applies: the reference arm informs the assumed post-discontinuation distribution.

Reference-Based MI With Time-to-Event Outcomes

Time-to-event endpoints require additional care.

Simple multivariate-normal reference-based imputation is generally not an appropriate direct solution for censored survival outcomes.

Instead, analysts may need methods designed around:

• Survival distributions
• Event times
• Censoring mechanisms
• Intercurrent events
• Risk-set information

The general principle of specifying a reference-based post-intercurrent-event scenario can still be useful, but the imputation model must respect the endpoint's structure.

Reference-Based MI and Non-Normal Continuous Outcomes

A continuous endpoint does not automatically justify a normal model.

If the outcome is highly skewed, bounded, ordinal, or otherwise non-normal, the analyst should consider an appropriate transformation or modeling strategy.

For example, a log-transformed biomarker may be modeled as:

with imputation performed on the transformed scale and results transformed back where appropriate.

Reference-Based MI and Intercurrent Events

Treatment discontinuation is one example of an intercurrent event.

Other examples include:

• Rescue medication
• Treatment switching
• Additional therapy
• Permanent discontinuation
• Protocol-defined treatment changes

The sensitivity strategy should make clear which intercurrent events are being addressed and how they affect the estimand.

Do Not Automatically Apply J2R to Every Missing Observation

A patient who misses one visit but returns at the next visit is fundamentally different from a patient who permanently discontinues treatment.

Therefore, a reference-based strategy should define the missingness patterns to which it applies.

For example:

Choosing Between J2R and CIR

The choice should be driven by the clinical question.

J2R may be appropriate when:

• Treatment benefit is assumed to disappear after discontinuation.
• The reference arm represents a plausible untreated trajectory.
• The sensitivity analysis is intended to test loss of treatment benefit.

CIR may be appropriate when:

• The benefit achieved before discontinuation is believed to persist.
• Subsequent changes are assumed to resemble the reference arm.
• The scientific scenario is better described by retaining the level achieved at discontinuation.

Neither assumption is universally more appropriate.

A Practical Decision Framework

A Compact Comparison

Simulation Is an Important Validation Tool

Simulation can be used to investigate whether the implementation behaves as expected.

For example, simulate data under a known longitudinal model, impose dropout, apply the reference-based imputation procedure, and compare the resulting estimates with the known truth under the simulated scenario.

A simple simulation structure is:

set.seed(12345)

n <- 400
M <- 100

# 1. Generate complete longitudinal data
# 2. Randomize treatment
# 3. Impose treatment discontinuation
# 4. Delete post-discontinuation outcomes
# 5. Apply reference-based MI
# 6. Analyze each completed dataset
# 7. Pool the estimates
# 8. Repeat many simulation replicates

Simulation can evaluate:

• Bias
• Coverage
• Type I error
• Power
• Monte Carlo variability
• Robustness to model misspecification

What Simulation Cannot Do

Simulation can validate an algorithm under a specified data-generating process.

It cannot prove that J2R or CIR is the correct assumption for a real clinical trial.

That is a scientific and clinical judgment.

Common Reporting Mistake: Calling the Sensitivity Analysis "Imputation of the Truth"

Reference-based imputation does not recover the true missing outcomes.

Instead, it creates plausible outcomes under an explicit assumption.

A better description is: "Missing outcomes were imputed under a jump-to-reference sensitivity scenario."

rather than: "Missing outcomes were reconstructed."

Another Common Mistake: Treating Sensitivity Analysis as an Afterthought

Reference-based sensitivity analyses should be planned before unblinding the results whenever possible.

The protocol or SAP should specify:

• The primary analysis
• The sensitivity estimands
• The reference treatment
• The reference-based method
• The relevant dropout patterns
• The imputation model
• The analysis model
• The pooling method

Clinical Interpretation of a Sensitivity Analysis

Suppose the primary analysis favors treatment.

There are several possibilities.

The Most Important Concept

Reference-based multiple imputation is best understood as a framework for making explicit assumptions about unobserved post-discontinuation outcomes.

The reference arm provides a clinically meaningful source of information for constructing those assumptions.

The major methods differ primarily in how much of the experimental-treatment trajectory is assumed to persist.

Multiple imputation then propagates uncertainty in those assumptions into the final treatment-effect inference.

Key Takeaways

• Reference-based MI uses another treatment arm to inform missing outcomes.
• J2R assumes the experimental treatment benefit does not continue after discontinuation in the reference-based sense.
• CIR retains the patient's achieved level but uses reference-arm increments thereafter.
• CR uses the reference trajectory/distribution according to the specified formulation.
• Reference-based MI is not the same as simple mean imputation.
• Multiple imputations are required to propagate missing-data uncertainty.
• Rubin's rules combine within- and between-imputation uncertainty.
• The covariance structure is an important component of longitudinal imputation.
• The reference-based assumption should be scientifically justified and prespecified.
• Reference-based sensitivity analyses are most useful when interpreted alongside the primary analysis and other plausible scenarios.

References

Carpenter, J.R., Roger, J.H. & Kenward, M.G. (2013). Analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible sensitivity analyses. Statistical Medicine, 32, 1683–1698.

Carpenter, J.R. & Kenward, M.G. (2013). Multiple Imputation and its Application. Wiley.

Little, R.J.A. & Rubin, D.B. Statistical Analysis with Missing Data. Wiley.

Rubin, D.B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley.

Rubin, D.B. (1987). Multiple imputation after 18+ years. Journal of the American Statistical Association, 91, 473–489.

White, I.R., Royston, P. & Wood, A.M. (2011). Multiple imputation using chained equations: issues and guidance for practice. Statistics in Medicine, 30, 377–399.

National Research Council (2010). The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press.

ICH E9(R1). Addendum on Estimands and Sensitivity Analysis in Clinical Trials. International Council for Harmonisation.