Sensitivity Analyses for Estimands

Introduction

Clinical trials rarely proceed exactly as planned. Patients may discontinue treatment, start rescue medication, switch treatments, experience an important clinical event, or miss outcome assessments.

These events create a fundamental statistical question: What treatment effect are we actually trying to estimate?

The ICH E9(R1) framework addresses this question through the concept of an estimand. An estimand specifies precisely what treatment effect is of interest by defining the population, treatment conditions, endpoint, handling of intercurrent events, and population-level summary measure.

Once an estimand has been defined, however, another question remains: How sensitive is the estimated treatment effect to assumptions required to estimate that estimand?

This is the role of sensitivity analysis.

Estimand vs. Estimator vs. Estimate

Three concepts should be kept separate.

For example, an estimand might be the difference in mean change from baseline at Week 24 under a treatment-policy strategy for treatment discontinuation.

An MMRM or another appropriate longitudinal model might be the estimator.

The resulting treatment difference, such as:

is the estimate.

What Is an Estimand?

Under ICH E9(R1), an estimand describes the treatment effect in a way that addresses five key attributes:

1. The population
2. The treatment condition being compared
3. The endpoint or variable of interest
4. How intercurrent events are handled
5. The population-level summary measure

For example, consider a randomized trial comparing Drug A with placebo using change from baseline in a continuous clinical score at Week 24.

Why Sensitivity Analysis Matters

Estimands often require assumptions that cannot be verified directly from the observed data.

This is particularly important when outcomes are missing after an intercurrent event or treatment discontinuation.

Suppose a patient stops treatment at Week 12 and therefore does not provide a Week 24 measurement.

If the target estimand is treatment policy, the scientific question concerns what would happen regardless of treatment discontinuation.

But the Week 24 outcome is not observed.

Some assumption or model is therefore needed to estimate the treatment-policy effect.

Primary Analysis vs. Sensitivity Analysis

A common mistake is to treat sensitivity analysis as an optional collection of alternative statistical models.

A strong estimand-based strategy instead starts with the scientific question, then identifies the assumptions required for the primary analysis.

The sensitivity analysis should challenge assumptions that could materially affect the conclusion.

Sensitivity Analysis Is Not the Same as Choosing Another Estimand

This distinction is especially important under ICH E9(R1).

Suppose the primary estimand uses a treatment-policy strategy for treatment discontinuation.

A hypothetical estimand asking what would have happened had treatment discontinuation not occurred is generally a different scientific question.

Therefore, simply replacing a treatment-policy analysis with a hypothetical analysis and calling it a "sensitivity analysis" can be misleading.

Intercurrent Events

An intercurrent event is an event occurring after treatment initiation that affects either the interpretation or existence of the outcome of interest.

Examples include:

• Permanent treatment discontinuation
• Use of rescue medication
• Treatment switching
• Death
• Organ transplantation
• Initiation of prohibited concomitant therapy
• Pregnancy or another protocol-defined event

The key issue is not merely that these events occur.

The key issue is: What treatment effect does the investigator want to describe after these events occur?

The Five ICH E9(R1) Strategies

ICH E9(R1) describes several strategies for handling intercurrent events.

Treatment Policy Strategy

Under a treatment-policy strategy, the intercurrent event does not remove the patient from the scientific question.

For example, if the patient discontinues study treatment, the estimand may still target the outcome that would be observed at Week 24 regardless of that discontinuation.

The scientific question might therefore be:

where the outcome is interpreted under the treatment policy rather than as an effect restricted to patients remaining on treatment.

Hypothetical Strategy

A hypothetical strategy asks a different question.

For example: What would the treatment effect have been if patients had not discontinued treatment?

This can be scientifically useful when the goal is to estimate the effect of continuous treatment under a specified hypothetical scenario.

However, outcomes under that scenario are generally not fully observable.

The analysis therefore relies on assumptions about what would have happened after the intercurrent event.

Composite Strategy

Under a composite strategy, the intercurrent event becomes part of the endpoint definition.

For example, suppose a trial evaluates time to clinical improvement and death is considered an important adverse outcome.

A composite endpoint might define:

The event is therefore not simply treated as missing information.

It changes the endpoint itself.

While-on-Treatment Strategy

A while-on-treatment strategy focuses on the treatment effect while patients remain exposed to the assigned treatment condition, according to a prespecified definition of the relevant treatment period.

This can be useful when the scientific question specifically concerns the effect while treatment is being taken.

However, restricting attention to observed treatment periods can introduce selection concerns if discontinuation is related to prognosis or treatment response.

Principal-Stratum Strategy

A principal-stratum strategy defines the target population according to potential behavior under treatment conditions.

For example, investigators might be interested in the treatment effect among patients who would not require rescue therapy under either treatment condition.

This is conceptually attractive but often requires strong assumptions because membership in the relevant principal stratum is not directly observable for each patient.

Where Sensitivity Analysis Enters the Estimand Framework

The estimand itself should generally be specified before the statistical analysis.

The sensitivity-analysis strategy then asks whether the estimated treatment effect remains credible under reasonable deviations from the assumptions needed to estimate that estimand.

Missing Data and Estimands

Missing data are one of the most important areas for estimand sensitivity analysis.

Consider a longitudinal endpoint:

If some Week 24 outcomes are missing, the observed data do not completely identify the distribution of the Week 24 endpoint without assumptions.

A primary model may make an assumption such as missing at random, while sensitivity analyses examine departures from that assumption.

Missing at Random

Under a missing-at-random framework, after conditioning on appropriate observed information, the probability of missingness does not depend on the unobserved outcome itself.

Informally:

where \(R\) denotes the observation indicator.

MAR can be a reasonable working assumption, but it cannot generally be verified from the observed data alone.

Missing Not at Random

A missing-not-at-random sensitivity analysis considers situations in which the probability of missingness depends on unobserved outcomes even after conditioning on observed information.

For example, patients who discontinue treatment because their unobserved outcome is substantially worse than their observed history might violate an MAR assumption.

The exact MNAR assumption must be made explicit.

There is no single universal "MNAR analysis."

Pattern-Mixture Models

Pattern-mixture models condition the outcome distribution on the pattern of missingness.

Let \(M\) denote a missingness pattern. The distribution can be represented conceptually as:

The unobserved components of \(f(Y\mid M)\) require assumptions.

A sensitivity analysis can vary those assumptions across missingness patterns.

Delta Adjustment

A common sensitivity-analysis technique is to start with an MAR-based imputation and then shift imputed values by a prespecified amount.

For a continuous outcome, this can be written as:

where \(\delta\) represents departure from the MAR-based prediction.

For example:

The treatment effect is recalculated across the delta values.

Why Delta Adjustment Is a Sensitivity Analysis

The scientific question has not necessarily changed.

Instead, the analysis asks: How much worse would the unobserved outcomes have to be for the conclusion to change?

This is exactly the type of assumption-based robustness question that a sensitivity analysis can address.

Reference-Based Imputation

Reference-based methods specify how outcomes after an intercurrent event or discontinuation relate to outcomes in another treatment group or reference trajectory.

Examples include approaches often described as:

• Jump to reference
• Copy reference
• Copy increments in reference
• Other prespecified reference-based assumptions

These approaches can be particularly useful for sensitivity analyses when investigators believe that post-discontinuation behavior could resemble a reference treatment trajectory.

Tipping Point Analysis

A tipping point analysis systematically varies assumptions about missing outcomes to identify where the clinical conclusion changes.

Suppose the primary analysis estimates a treatment difference of:

with a confidence interval excluding zero.

The investigator may vary assumptions about missing outcomes until the confidence interval first includes zero.

The resulting boundary is the approximate tipping point.

A Simple Tipping-Point Example

Suppose the treatment effect is evaluated under progressively more pessimistic assumptions about missing outcomes.

The tipping point lies between the small and moderate departures.

The key interpretation is not simply that the sensitivity analysis "failed."

Rather, the result tells investigators how substantial the departure from the primary assumption would need to be before the inferential conclusion changes.

Statistical Significance Is Not the Only Tipping Point

A clinically meaningful tipping point may be more informative than a purely statistical one.

Suppose the clinically important treatment difference is:

A sensitivity analysis could ask when the estimated treatment effect falls below 3 rather than when its confidence interval crosses zero.

This can produce a more clinically relevant assessment of robustness.

Worked Example: Continuous Endpoint

Consider a randomized clinical trial with a continuous endpoint measured at Week 24.

The primary estimand is:

• Population: randomized eligible patients
• Treatment: Drug A vs placebo
• Endpoint: Week 24 change from baseline
• Intercurrent event: treatment discontinuation handled using treatment policy
• Summary measure: difference in mean change

Suppose the primary analysis produces:

with:

The treatment effect is therefore positive and the confidence interval excludes zero.

Why a Sensitivity Analysis Is Still Needed

Suppose 18% of patients do not have a Week 24 measurement.

The primary estimator requires assumptions about those missing outcomes.

Even if the primary analysis is statistically valid under its assumptions, the robustness of the conclusion should be explored.

One possible sensitivity analysis applies increasingly pessimistic delta adjustments to missing outcomes.

Delta-Adjusted Sensitivity Analysis

Suppose the analysis considers:

The resulting estimates might be:

The statistical conclusion changes around a delta of approximately -3.

The interpretation is: the primary conclusion is reasonably robust unless missing outcomes are assumed to be approximately three or more units worse than the MAR-based predictions.

Sensitivity Analysis for Binary Endpoints

Sensitivity analysis is not limited to continuous outcomes.

Suppose the endpoint is remission at Week 24.

The treatment effect might be expressed as a risk difference:

or a risk ratio:

Missing binary outcomes may require assumptions about the probability of response among patients whose outcomes are unobserved.

Binary Tipping-Point Analysis

For a binary endpoint, one can vary the assumed response probabilities among patients with missing outcomes.

For example, suppose the sensitivity analysis varies:

• Missing treatment-arm response probability from 10% to 50%
• Missing control-arm response probability from 10% to 50%

The analysis then evaluates where the treatment conclusion changes.

This provides an intuitive picture of how assumptions about missing outcomes affect the treatment comparison.

Intercurrent Events and Rescue Medication

Rescue medication is a particularly important example.

Suppose patients who fail treatment can receive rescue therapy.

Several scientifically different estimands are possible.

These are not merely alternative statistical methods.

They correspond to different scientific questions.

Sensitivity Analysis After a Treatment-Policy Estimand

Suppose treatment policy is the primary strategy for rescue medication.

The sensitivity analysis might investigate whether the treatment-policy conclusion is robust to assumptions about missing outcomes after rescue.

For example:

• Primary MAR-based analysis
• Delta-adjusted analysis
• Pattern-mixture sensitivity analysis
• Reference-based sensitivity analysis
• Worst-case or conservative scenarios where clinically defensible

A hypothetical estimand may also be reported as a separate estimand if it answers a scientifically important question, but it should not automatically be described as a sensitivity analysis for the treatment-policy estimand.

Death Requires Special Attention

Death is often qualitatively different from ordinary missing data.

Suppose the endpoint is a continuous functional outcome at Week 24.

A patient who dies before Week 24 does not simply have an inconveniently missing measurement.

Death may fundamentally affect the meaning of the endpoint.

Possible strategies include:

• Composite strategy incorporating death
• Hypothetical strategy under a specified counterfactual scenario
• Other estimand-specific approaches justified by the clinical question

Reference-Based Sensitivity Analysis Example

Consider a two-arm longitudinal study.

Patients who discontinue active treatment may have no future measurements.

A reference-based sensitivity analysis could assume that after discontinuation, their future mean trajectory follows a reference-arm trajectory according to a prespecified rule.

The imputed outcome might conceptually be represented as:

The exact implementation depends on the longitudinal model and reference-based assumption.

The sensitivity analysis then compares the treatment effect with the primary analysis.

How to Judge Robustness

A sensitivity analysis should not be reduced to the question: Is the p-value still below 0.05?

A more complete assessment considers:

• Direction of the treatment effect
• Magnitude of the treatment effect
• Confidence interval
• Clinical relevance
• Statistical significance
• Consistency across plausible assumptions
• Magnitude of departure from the primary assumption required to reverse the conclusion

A Robustness Matrix

A useful way to summarize sensitivity analyses is a robustness matrix.

This format allows reviewers to see exactly which assumptions drive the conclusion.

What Makes a Good Sensitivity Analysis?

A useful sensitivity analysis should satisfy four principles.

1. It Should Be Relevant

The analysis should challenge an assumption that could plausibly affect the primary conclusion.

2. It Should Be Prespecified

The major sensitivity analyses should be described before unblinding the comparative treatment results whenever possible.

3. It Should Be Plausible

The alternative assumptions should have a clinical or statistical rationale.

4. It Should Be Interpretable

The analysis should make clear what changed and why.

Bad Sensitivity Analysis

Consider a study in which the primary analysis favors treatment.

After seeing the results, investigators try ten alternative models until one produces a non-significant result.

This is not a strong sensitivity-analysis strategy.

The problem is not that alternative analyses were performed.

The problem is that the alternatives were selected opportunistically after seeing the treatment effect.

Sensitivity Analysis vs. Subgroup Analysis

These concepts are also different.

A subgroup analysis can itself have sensitivity considerations, but it is not automatically a sensitivity analysis.

Sensitivity Analysis vs. Per-Protocol Analysis

A per-protocol analysis is often incorrectly described as a sensitivity analysis for the primary treatment-policy estimand.

Whether it is appropriate depends on the scientific question.

Restricting the analysis to patients who adhered sufficiently to treatment can change the population being analyzed and can introduce selection.

If the per-protocol effect is scientifically important, it may be better described explicitly as a separate estimand or supportive analysis rather than simply labeling it a sensitivity analysis.

Sensitivity Analysis and Treatment Adherence

Suppose the primary estimand asks: What is the treatment-policy effect of assignment to Drug A versus placebo?

An analysis restricted to adherent patients asks a different question: What is the treatment effect among patients satisfying a particular adherence condition?

The second question may be valuable, but it should not be confused with the first.

Sensitivity Analysis for Time-to-Event Endpoints

Sensitivity analysis also plays an important role in survival analysis.

Suppose the primary endpoint is time to disease progression or death.

Intercurrent events may include:

• Subsequent anticancer therapy
• Treatment switching
• Discontinuation
• Loss to follow-up
• Death before a nonfatal endpoint

The estimand must define how these events are handled.

Sensitivity analyses can then investigate assumptions about censoring, subsequent therapy, treatment switching, or other relevant events.

Example: Subsequent Therapy

Suppose the endpoint is time to progression.

A treatment-policy strategy might consider progression regardless of subsequent therapy, provided the endpoint remains observable and is defined appropriately.

A hypothetical strategy might ask what progression would have been under a specified scenario without subsequent therapy.

A sensitivity analysis might investigate how assumptions about progression after subsequent therapy affect the estimated treatment effect.

Again, the critical distinction is between:

and:

Formalizing Sensitivity Analysis

Let the primary estimand be:

and let the primary estimator depend on an assumption indexed by \(\delta\):

where:

represents the primary assumption.

A sensitivity analysis evaluates:

The goal is to understand how much the estimated treatment effect changes as the assumption changes.

Confidence Intervals Matter

Suppose the estimated treatment effect remains positive across all sensitivity analyses.

That is useful evidence of robustness.

But the width of the confidence interval may change substantially.

For example:

The treatment effect remains positive, but inferential certainty has weakened.

A complete interpretation should report both aspects.

Clinically Meaningful Robustness

Suppose the minimum clinically important treatment difference is:

If all plausible sensitivity analyses produce estimates above 3, the result may be considered robust from a clinical perspective even if some confidence intervals cross zero.

Conversely, if modest assumptions reduce the estimate below 3, the clinical interpretation may be sensitive even when statistical significance remains.

What Should Be Prespecified?

The statistical analysis plan should describe the sensitivity-analysis strategy in enough detail that the analyses can be reproduced.

Important elements include:

• Primary estimand
• Primary estimator
• Intercurrent-event strategies
• Missing-data assumptions
• Variables included in the model
• Alternative assumptions to be investigated
• Range of sensitivity parameters
• Imputation procedure
• Pooling procedure for multiple imputation
• Confidence-interval method
• Decision criteria for interpreting robustness

Sensitivity Parameters

A sensitivity parameter is a parameter that controls departure from an identifying assumption.

For example:

may represent the difference between an MAR-based prediction and a specified MNAR scenario.

A sensitivity analysis then evaluates the treatment effect over a range:

The range should be justified.

One-Way Sensitivity Analysis

A one-way sensitivity analysis changes one assumption at a time.

For example:

while keeping the other modeling assumptions fixed.

This is easy to interpret and often useful for identifying a tipping point.

Multi-Dimensional Sensitivity Analysis

Some clinical trial problems involve several uncertain assumptions.

For example:

• Missing outcomes may be worse than MAR predicts.
• Patients after treatment discontinuation may behave differently from completers.
• Rescue medication may change subsequent outcomes.

A more comprehensive sensitivity analysis can vary multiple assumptions simultaneously.

This can be represented as:

The resulting analysis can be displayed as a grid of treatment effects or confidence intervals.

Why Multiple Imputation Is Useful

When missing outcomes are imputed multiple times, uncertainty about the missing values can be incorporated into the variance estimate.

Suppose \(m\) imputed datasets are generated.

Each dataset produces:

where \(U_j\) is the estimated within-imputation variance.

The pooled estimate is:

Rubin's combining rules then account for both within-imputation and between-imputation variability.

Sensitivity Analysis After Multiple Imputation

Multiple imputation does not eliminate the need for sensitivity analysis.

The imputation model itself contains assumptions.

For example, an MAR-based multiple-imputation model might be followed by MNAR sensitivity analyses using delta adjustments.

The resulting framework is:

R Example: Delta-Adjusted Sensitivity Analysis

The following simplified R code illustrates the conceptual structure of a delta-adjusted analysis.

deltas <- c(0, -1, -2, -3, -4, -5)

results <- data.frame(
  delta = deltas,
  estimate = NA_real_
)

for (i in seq_along(deltas)) {

  delta <- deltas[i]

  # Apply delta adjustment to
  # imputed missing outcomes

  y_imp_sens <- y_imp_mar + delta

  # Fit the analysis model

  fit <- lm(
    change ~ treatment + baseline,
    data = y_imp_sens
  )

  results$estimate[i] <-
    coef(fit)["treatment"]
}

results

In an actual multiple-imputation analysis, the implementation should repeat the imputation, analysis, and pooling procedures appropriately for each sensitivity scenario.

R Example: Tipping-Point Grid

A two-dimensional tipping-point analysis can be represented using a grid of sensitivity parameters.

delta_trt <- seq(-5, 0, by = 1)
delta_ctl <- seq(-5, 0, by = 1)

grid <- expand.grid(
  delta_trt = delta_trt,
  delta_ctl = delta_ctl
)

grid$estimate <- NA_real_

for (i in seq_len(nrow(grid))) {

  # Apply the treatment-arm
  # and control-arm assumptions

  # Re-impute and analyze here

  grid$estimate[i] <- NA_real_
}

grid

A production implementation would replace the placeholder with the complete multiple-imputation and analysis workflow.

SAS Example: Sensitivity Parameter Framework

The exact implementation depends on the selected methodology, but a sensitivity parameter can be incorporated into an imputation workflow.

%let delta = -2;

/* Example conceptual workflow */

/* 1. Generate MAR-based imputations */

/* 2. Apply delta adjustment */

/* 3. Analyze each imputed dataset */

/* 4. Pool estimates */

/* 5. Repeat for each prespecified delta */

For validated clinical-trial programming, the final implementation should include dataset-level traceability, treatment-arm definitions, analysis populations, derivation rules, imputation specifications, and pooling procedures consistent with the SAP.

Common Sensitivity Analyses for Estimands

Alternative Estimands as Supportive Analyses

Sometimes investigators want to understand how conclusions differ across scientifically relevant estimands.

For example, a trial might define:

• A primary treatment-policy estimand
• A hypothetical estimand
• A while-on-treatment estimand

Comparing these estimands can be highly informative.

However, the results should be labeled according to the scientific questions they answer.

This is clearer than simply saying: "A hypothetical sensitivity analysis was performed."

Estimand Sensitivity Analysis Matrix

A useful planning tool is an estimand-analysis matrix.

The Role of Assumptions

Every sensitivity analysis should identify the assumption being challenged.

For example:

This makes the sensitivity analysis scientifically interpretable.

Do Not Overload the Analysis With Arbitrary Scenarios

More sensitivity analyses are not automatically better.

An SAP containing dozens of arbitrary alternative models can make the primary conclusion harder to interpret.

A better strategy is to identify the assumptions that matter most and construct a focused set of clinically defensible sensitivity analyses.

Three Levels of Sensitivity

A practical framework is to think about sensitivity analysis at three levels.

The third category should be clearly distinguished from sensitivity analyses that retain the same estimand.

Worked Example: Estimand and Sensitivity Strategy

Consider a randomized trial of Drug A versus placebo with a continuous symptom score measured through Week 24.

Patients may discontinue treatment and may receive rescue medication.

The primary estimand is defined as:

Primary Analysis

Suppose the primary MMRM produces:

with:

Sensitivity Analysis 1: Delta Adjustment

The analysis varies the post-discontinuation missing outcomes by:

Suppose all scenarios continue to show a positive treatment effect.

This supports robustness to the corresponding range of missing-data departures.

Sensitivity Analysis 2: Reference-Based Imputation

A reference-based approach is used to assess a substantially different post-discontinuation trajectory assumption.

Suppose the estimated effect is:

The estimate is smaller but remains clinically meaningful.

Sensitivity Analysis 3: Hypothetical Estimand

A separate hypothetical estimand asks: What would the Week 24 treatment effect be if patients had remained on their assigned treatment without rescue medication?

Suppose that analysis produces:

This result should not simply be interpreted as another estimate of the treatment-policy estimand.

It answers a different question.

Putting the Results Together

This presentation is much more informative than labeling every result a "sensitivity analysis."

Common Mistakes

1. Confusing an estimand with an analysis method. An estimand is the treatment-effect quantity of interest, not the name of the statistical model.
2. Calling every alternative analysis a sensitivity analysis. Some alternatives answer different scientific questions and are better described as separate estimands.
3. Treating MAR as proven. MAR is generally an assumption, not something established by a statistical test of the observed data.
4. Using arbitrary MNAR shifts. Sensitivity parameters should be clinically or statistically justified.
5. Ignoring intercurrent events. Treatment discontinuation, rescue therapy, switching, and death can change the meaning of the treatment effect.
6. Reporting only p-values. The magnitude and uncertainty of the treatment effect should also be compared.
7. Performing sensitivity analyses only after seeing the treatment effect. Major sensitivity analyses should be prespecified whenever possible.
8. Calling a per-protocol analysis automatically a sensitivity analysis. A per-protocol analysis may target a different population or scientific question.
9. Ignoring the plausibility of the tipping point. A result can be statistically fragile under an extreme assumption but robust under all clinically plausible assumptions.
10. Failing to distinguish missingness from an intercurrent event. An intercurrent event is a clinical event; missingness is an observation-data problem. They may interact, but they are not the same concept.
11. Changing the estimand unintentionally. Restricting the analysis population or changing endpoint definitions can change the scientific question.
12. Using "MNAR" as if it were one specific analysis. MNAR encompasses many possible assumptions. The actual sensitivity model must be specified.

A Practical Workflow

What Should Be in the Statistical Analysis Plan?

A strong SAP should make the sensitivity-analysis strategy reproducible.

At minimum, specify:

• Primary estimand
• Primary analysis population
• Primary estimator
• Intercurrent-event strategies
• Primary missing-data assumptions
• Variables used in the analysis model
• Sensitivity-analysis estimands, if separate estimands are evaluated
• Sensitivity parameters
• Range of parameter values
• Imputation methodology
• Reference-based assumptions, if applicable
• Number of imputations
• Pooling methodology
• Confidence-interval methodology
• Tipping-point methodology
• Criteria for evaluating robustness

How to Report Sensitivity Analyses

A useful clinical-trial report should not simply state:

"Sensitivity analyses supported the primary analysis."

Instead, report what assumptions were changed and how the treatment effect responded.

For example:

What Regulators and Reviewers Want to Understand

A sensitivity-analysis section should make it possible to answer:

• What was the primary scientific question?
• What was the primary estimand?
• What intercurrent events occurred?
• How were those events handled?
• Which assumptions were necessary?
• Which assumptions were varied?
• Were the alternative assumptions plausible?
• Did the treatment effect change materially?
• Did the clinical conclusion change?
• How extreme did the assumptions have to become before the conclusion changed?

The Difference Between Robust and Fragile Conclusions

Consider two trials.

In Trial A, the primary estimate is 5.0 units. The estimate remains between 4.2 and 5.0 across all clinically plausible sensitivity analyses.

This is evidence of robustness.

In Trial B, the primary estimate is 5.0 units, but a modest and clinically plausible departure from the primary missing-data assumption reduces the estimate to 1.0 unit.

The second conclusion is much more sensitive to assumptions.

Sensitivity Analysis Is About Uncertainty

Standard confidence intervals quantify sampling uncertainty conditional on the model and assumptions.

Sensitivity analysis addresses a different source of uncertainty: uncertainty about assumptions.

Conceptually, there are therefore at least two layers of uncertainty:

This expression is conceptual rather than a literal variance decomposition in all settings.

The important idea is that a narrow confidence interval does not guarantee a robust conclusion if the underlying assumptions are highly uncertain.

Why the Estimand Comes First

It is tempting to begin with a familiar statistical model:

• MMRM
• ANCOVA
• Multiple imputation
• Cox regression
• Mixed-effects model

and then ask how to handle missing data.

The estimand framework reverses this order.

First ask: What treatment effect do we want to know?

Then determine: What assumptions and statistical procedures are needed to estimate that effect?

Finally ask: How sensitive is the conclusion to those assumptions?

Key Takeaways

• An estimand defines the treatment effect that the clinical trial seeks to answer.
• A sensitivity analysis evaluates how robust that conclusion is to plausible assumptions or estimation choices.
• Changing the estimand is not necessarily the same thing as performing a sensitivity analysis.
• Intercurrent events should be addressed explicitly through a prespecified strategy.
• Missing data are especially important because estimating many estimands requires assumptions about unobserved outcomes.
• MAR is an assumption, not a guarantee established by the observed data.
• Delta adjustment provides a transparent way to investigate departures from an MAR-based analysis.
• Pattern-mixture models provide a framework for sensitivity analyses involving missing-data patterns.
• Reference-based methods can explore clinically motivated assumptions about post-discontinuation trajectories.
• Tipping-point analyses show how extreme an assumption must become before a clinical or statistical conclusion changes.
• Confidence intervals and clinical relevance should be considered in addition to p-values.
• Major sensitivity analyses should be prespecified whenever possible.

Bottom Line

References

International Council for Harmonisation (ICH). E9(R1): Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials.

ICH. E9: Statistical Principles for Clinical Trials.

National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press.

Little, R.J.A., D'Agostino, R., Cohen, M.L., et al. The prevention and treatment of missing data in clinical trials. New England Journal of Medicine.

Mallinckrodt, C.H., Lane, P.W., Schnell, D., Peng, Y., Mancuso, J.P. Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials.

Carpenter, J.R. & Kenward, M.G. Multiple Imputation and its Application. Wiley.

Cro, S., Morris, T.P., Kenward, M.G., Carpenter, J.R. Reference-based sensitivity analysis via multiple imputation for longitudinal clinical trials.