Understanding ICH E9 Statistical Principles

Introduction

ICH E9: Statistical Principles for Clinical Trials is one of the core statistical guidelines used when planning, analyzing, and reporting confirmatory clinical trials. Its purpose is not to prescribe one statistical test for every study. Instead, it provides a framework for making statistical choices that are scientifically justified, prospectively specified, and aligned with the clinical question.

The original E9 guideline emphasizes the principles underlying trial design, conduct, analysis, and interpretation. The later ICH E9(R1) addendum strengthens that framework by introducing the concept of the estimand and by explicitly connecting the clinical question to intercurrent events, the estimator, the resulting estimate, and sensitivity analyses.

The ICH E9 Mental Model

A useful way to understand E9 is to think of a clinical trial as a chain of decisions:

The key conceptual separation is therefore:

What Is ICH E9?

ICH E9 provides statistical principles for the design, conduct, analysis, and interpretation of clinical trials. It addresses topics such as trial objectives, design features, endpoints, analysis populations, sample-size determination, interim analyses, missing data, statistical methods, multiplicity, and presentation of results.

It is especially important in confirmatory trials because decisions made before database lock can strongly affect the credibility of the treatment-effect estimate.

ICH E9 and ICH E9(R1)

The original E9 guideline and E9(R1) should be read together. E9(R1) does not replace the broader statistical principles in E9. Instead, it adds a framework for defining treatment effects precisely through estimands and for aligning trial design and analysis with those targets.

The Fundamental Distinction: Estimand, Estimator, and Estimate

These terms sound similar but represent different concepts.

Estimand

An estimand is the precise target of estimation: the treatment effect corresponding to the clinical question.

Estimator

An estimator is the statistical procedure or rule used to obtain an estimate of the estimand.

Estimate

An estimate is the numerical result produced from the observed data.

For example, a trial may define the estimand as the treatment difference in mean change from baseline at Week 24 under a treatment-policy strategy for treatment discontinuation. An ANCOVA or an appropriate longitudinal model might serve as the estimator, producing a numerical treatment difference such as −4.8 units with a confidence interval.

Why the Estimand Matters

Consider a randomized trial of an investigational treatment for a chronic disease. Some participants discontinue treatment because of adverse events, some start rescue medication, and some remain on treatment through Week 24.

There is no single automatically correct answer to the question, “What is the treatment effect at Week 24?” The answer depends on what the clinical question intends to do with those events.

For example, do we want to know:

• What would happen if patients followed the treatment strategy regardless of discontinuation?
• What would happen while patients remained on assigned treatment?
• What would happen if rescue medication were prevented?
• What is the treatment effect if a particular intercurrent event occurs?

Those are different scientific questions and can correspond to different estimands.

The Five Attributes of an Estimand

ICH E9(R1) describes an estimand using five attributes:

Intercurrent Events

An intercurrent event is an event occurring after treatment initiation that affects either the interpretation or the existence of the measurements associated with the clinical question.

Common examples include:

• Permanent treatment discontinuation
• Temporary treatment interruption
• Rescue medication
• Additional or prohibited therapy
• Treatment switching
• Death before the planned endpoint
• Organ transplantation or another event that changes the outcome process

The important point is that an intercurrent event is not merely a nuisance missing-data flag. It can change what treatment effect is scientifically meaningful.

The Five Intercurrent-Event Strategies

ICH E9(R1) describes five strategies for addressing intercurrent events.

These strategies should not be treated as interchangeable statistical tricks. Each represents a different scientific interpretation of the treatment effect.

A Simple Estimand Example

Suppose a study compares Drug A with placebo on a continuous symptom score at Week 24. Rescue medication is available if symptoms worsen.

A treatment-policy estimand might be expressed conceptually as:

where the Week 24 outcome is considered regardless of rescue medication, consistent with the treatment-policy question.

A hypothetical estimand could instead ask what the Week 24 treatment difference would have been if rescue medication had not been available.

Randomization: The Foundation of Comparative Inference

Randomization is one of the most important design principles in controlled clinical trials. Proper random allocation makes treatment assignment independent of measured and unmeasured baseline characteristics in expectation.

For a two-arm trial, the treatment effect can be conceptualized as a comparison of outcomes under the two randomized treatment conditions:

where \(\mu_T\) and \(\mu_C\) denote the relevant population-level outcome summaries under treatment and control.

Randomization does not guarantee that the two observed groups will have identical baseline characteristics. It provides the probabilistic basis for valid comparison.

Stratified Randomization

When important prognostic factors are known before randomization, stratification can help maintain balance across treatment groups.

Examples include:

• Disease severity
• Geographic region
• Prior treatment status
• Biomarker category

Stratification should be reflected consistently in the randomization scheme and, where appropriate, in the primary analysis model.

Blinding and Control Groups

Blinding reduces the opportunity for knowledge of treatment assignment to influence patient behavior, investigator assessment, outcome measurement, or other aspects of trial conduct.

A well-selected control group provides the counterfactual comparison needed to distinguish treatment effects from background changes such as natural history, placebo response, regression to the mean, or changes in concomitant care.

Choosing the Primary Endpoint

The primary endpoint should be closely connected to the main clinical objective. It should be sufficiently sensitive to the treatment effect of interest and measured using procedures that minimize avoidable variability and bias.

Important considerations include:

• Clinical relevance
• Reliability and validity of measurement
• Timing of assessment
• Expected treatment effect
• Variability
• Missingness
• Interpretability
• Multiplicity implications when multiple endpoints are considered

Continuous, Binary, and Time-to-Event Endpoints

The treatment-effect measure should be selected because it answers the clinical question clearly, not simply because it is familiar to the programmer.

Hypothesis Testing

For many confirmatory trials, a primary analysis is framed using a null hypothesis and an alternative hypothesis.

For a two-sided test at significance level \(\alpha\), the decision rule is designed to control the probability of a false-positive conclusion under the null according to the prespecified testing framework.

For a simple normally distributed mean comparison, a standardized test statistic might take the form:

In practice, clinical trials may use ANCOVA, mixed models, generalized linear models, survival models, exact methods, rank-based methods, or other estimators depending on the endpoint and design.

P-Values and Confidence Intervals

A p-value quantifies how incompatible the observed data are with a specified null hypothesis under the assumptions of the statistical test. It is not the probability that the null hypothesis is true.

A confidence interval provides a range of treatment-effect values compatible with the data and the interval procedure's assumptions.

Statistical Significance Is Not Clinical Importance

A very large trial can detect a small treatment difference with a small p-value. Conversely, a clinically meaningful effect can fail to reach statistical significance in an underpowered study.

A useful interpretation therefore considers:

• Magnitude of the treatment effect
• Confidence interval
• Clinical relevance
• Prespecified decision criteria
• Consistency across analyses and endpoints
• Safety and benefit-risk context

Sample Size and Power

Sample size should be based on the statistical design and the treatment effect the trial is intended to detect.

For a simple two-arm comparison of means with equal allocation, a conceptual sample-size relationship is:

where \(\sigma\) is the assumed common standard deviation, \(\Delta\) is the target treatment difference, \(\alpha\) is the type I error rate, and \(1-\beta\) is power.

This formula is illustrative. Actual sample-size calculations should reflect the planned endpoint, analysis model, allocation, dropout assumptions, interim design, multiplicity strategy, and other protocol-specific features.

Power Is a Design Quantity

Power is the probability that a specified statistical procedure rejects the null hypothesis under a particular alternative scenario and set of assumptions.

Power is not a property of the drug alone. It depends on the assumed effect size, variability, sample size, significance level, design, and analysis method.

Type I and Type II Error

ICH E9 emphasizes controlling the probability of false-positive conclusions in confirmatory testing while designing the study with adequate power for the scientifically important effect.

Multiplicity

Multiplicity arises when multiple statistical opportunities can produce a false-positive conclusion. Examples include:

• Multiple primary endpoints
• Multiple doses
• Multiple treatment comparisons
• Multiple time points used for formal testing
• Multiple subgroups used for confirmatory claims
• Interim looks at accumulating data

If multiple hypotheses are tested without an appropriate strategy, the probability of at least one false-positive conclusion can exceed the nominal type I error rate.

Common approaches include hierarchical testing, gatekeeping, alpha allocation, and other prespecified multiplicity-control procedures.

Example: Multiple Hypotheses

Suppose three independent hypotheses are each tested at \(\alpha=0.05\). Under a simplified independence assumption, the probability of making at least one false rejection when all three null hypotheses are true is:

This is approximately 14.3%, not 5%.

Interim Analyses

Interim analyses can be valuable for ethical, operational, or scientific reasons, but they create statistical considerations because the accumulating data can provide repeated opportunities to stop or claim success.

Examples include:

• Early stopping for overwhelming efficacy
• Early stopping for futility
• Sample-size reassessment
• Safety monitoring
• Adaptive design decisions

When interim analyses are used for confirmatory decisions, the analysis plan should specify the timing, decision boundaries or alpha-spending approach, information fraction where relevant, and how the final inference will account for the interim looks.

Missing Data: A Statistical Problem, Not Just a Programming Problem

Missing observations can occur because of treatment discontinuation, adverse events, loss to follow-up, withdrawal of consent, logistical problems, or other reasons.

The key issue is not merely how many observations are missing. The reason for missingness and its relationship to treatment and outcome can affect the validity of the treatment-effect estimate.

MCAR, MAR, and MNAR

These are assumptions about the missing-data mechanism, not labels that can be established merely by inspecting a missing-data percentage.

Why “Last Observation Carried Forward” Is Not a General Solution

LOCF is sometimes tempting because it creates a complete dataset. But carrying the last observed value forward imposes a strong assumption about the future outcome trajectory and can distort variability and treatment-effect estimates.

The preferred approach should be justified by the estimand, endpoint, data structure, and assumptions rather than chosen merely because it produces a complete table.

Sensitivity Analysis

A primary analysis is based on assumptions. Sensitivity analyses examine whether the conclusion is robust to plausible departures from those assumptions or to reasonable alternative analytical approaches.

Examples include:

• Alternative missing-data assumptions
• Pattern-mixture models
• Reference-based imputation
• Tipping-point analyses
• Alternative covariance structures
• Alternative definitions of the analysis population
• Alternative handling of intercurrent events

A useful sensitivity analysis should address an uncertainty that could plausibly affect the interpretation of the trial.

The Tipping-Point Idea

Suppose the primary analysis favors treatment. A tipping-point analysis can ask how extreme the assumptions about missing outcomes would have to become before the conclusion changes.

Conceptually, let:

where \(d\) represents a systematic departure imposed on the missing outcomes. The analyst can examine the range of \(d\) values under which the treatment conclusion changes.

Analysis Populations

Clinical trials often describe several analysis populations. The names can vary by protocol, but common concepts include:

The exact definitions should be prespecified. They should not be altered after seeing which participants produce the most favorable result.

Why Randomized Analysis Matters

For a superiority trial, excluding participants after randomization based on post-randomization behavior can compromise the comparability created by randomization.

This is why the primary analysis population and estimand should be aligned before the study begins.

Covariate Adjustment

ICH E9 supports the use of appropriate covariates when justified. Covariate adjustment can improve precision and account for important prognostic factors.

For a continuous endpoint, an ANCOVA model might be written as:

where \(T_i\) is treatment assignment and \(B_i\) is a prespecified baseline covariate.

The treatment coefficient should be interpreted according to the model and estimand, not automatically as an unadjusted difference in means.

Baseline Covariates: Precision Versus Post-Randomization Adjustment

Baseline prognostic covariates are generally safe candidates for adjustment when prespecified and appropriately measured. Post-randomization variables are different because they may be affected by treatment and conditioning on them can introduce bias.

Subgroup Analyses

Subgroup analyses can be scientifically valuable, particularly when there is a biologically plausible reason to expect treatment-effect heterogeneity.

But an apparent difference in subgroup-specific p-values does not itself establish interaction.

A treatment-by-subgroup interaction model is often more appropriate:

The interaction term \(\beta_3\) addresses whether the treatment effect differs across subgroup levels under the specified model.

Regression to the Mean

Participants are often enrolled because their disease measure is unusually high or low. Random fluctuation can cause subsequent measurements to move closer to the population average even without an effective treatment.

A randomized control group helps distinguish regression to the mean from treatment benefit.

Endpoint Timing

The timing of the primary endpoint should be clinically justified. Choosing a time point solely because it produces the most favorable result undermines the confirmatory interpretation.

Examples include:

• Week 12 response
• Week 24 change from baseline
• Time to progression
• Event-free survival at a specified time

When treatment effects evolve over time, the endpoint definition and estimand should make the intended time horizon explicit.

Time-to-Event Analysis

For a time-to-event endpoint, the survival function can be represented as:

Common analyses include Kaplan-Meier estimation and Cox proportional-hazards models.

A Cox model can be expressed as:

with a hazard ratio of:

The hazard ratio is not generally the same thing as a ratio of probabilities or a ratio of median survival times. Interpretation must match the estimand and model.

Restricted Mean Survival Time

When the proportional-hazards assumption is questionable or when a time-window-based summary is clinically attractive, restricted mean survival time can be useful.

The treatment effect can then be expressed as a difference in RMST over a prespecified horizon \(\tau\).

Protocol, SAP, and TLF Traceability

ICH E9 principles are ultimately operationalized through protocol objectives, the statistical analysis plan, programming specifications, and the final tables, listings, and figures.

Data Standards and the E9 Framework

ICH E9 is not a data-standard document, but statistical principles interact directly with modern clinical-trial data standards.

The statistical programmer should be able to trace a primary treatment-effect estimate back through the analysis dataset to the underlying source observations and prespecified derivations.

R Example: Continuous Endpoint

Suppose the primary endpoint is change from baseline at Week 24. A simple ANCOVA implementation might look like this:

model <- lm(
  CHG24 ~ TRT01P + BASE,
  data = analysis_data
)

summary(model)

# Estimated treatment effect
coef(model)["TRT01PDrug A"]

In a real clinical trial, the model would need to reflect the protocol and SAP, including treatment coding, stratification factors where applicable, covariance assumptions, missing-data handling, and the defined estimand.

R Example: Binary Endpoint

fit <- glm(
  RESPONSE ~ TRT01P + BASE,
  data = analysis_data,
  family = binomial()
)

summary(fit)

The model coefficient is on the log-odds scale. If the clinical estimand is a risk difference or risk ratio, the model and post-estimation procedure should be chosen accordingly rather than assuming that an odds ratio is automatically the desired treatment effect.

R Example: Time-to-Event Endpoint

library(survival)

fit <- coxph(
  Surv(AVAL, CNSR) ~ TRT01P + STRATA,
  data = analysis_data
)

summary(fit)

The exact event and censoring definitions should come from the prespecified endpoint definition and estimand.

SAS Example: ANCOVA

proc glm data=analysis_data;
    class TRT01P;
    model CHG24 = TRT01P BASE;
    lsmeans TRT01P / pdiff cl;
run;
quit;

For regulated clinical-trial programming, the production implementation should also include appropriate dataset qualification, analysis flags, treatment coding, format management, output validation, and traceability.

SAS Example: Logistic Regression

proc logistic data=analysis_data;
    class TRT01P(ref="Placebo") / param=ref;
    model RESPONSE(event="1") = TRT01P BASE;
    lsmeans TRT01P / ilink diff cl;
run;

Again, the choice of effect scale should be driven by the estimand and SAP rather than by the convenience of the model output.

Prespecification

One of the recurring themes in E9 is that important statistical decisions should be specified before unblinding or before access to comparative treatment results, as appropriate.

Prespecification reduces the opportunity for analyses to be selected because they produce a preferred answer.

Protocol Deviations

Protocol deviations should be assessed according to predefined criteria. The existence of a deviation does not automatically mean that a participant should be excluded from every analysis.

A per-protocol analysis can be informative as a supportive analysis, but excluding participants after observing outcomes can introduce selection bias.

Outliers

An extreme observation should not be removed simply because it changes the p-value.

Instead, investigate:

• Whether the value is a data-entry error
• Whether the measurement is scientifically plausible
• Whether the protocol specifies a rule for handling it
• Whether the model is sensitive to the observation
• Whether robust or alternative analyses are appropriate

Transformations

Transformations can be useful when the endpoint distribution is strongly skewed or when model assumptions are better satisfied on a transformed scale.

For a positive endpoint, a log transformation is:

Interpretation must then be translated back to the clinically meaningful scale when appropriate.

Assumptions Should Be Visible

A statistical result is only as credible as the assumptions needed to interpret it.

Primary Analysis Versus Sensitivity Analysis

Common Misunderstanding: Sensitivity Analysis Is Not a “Second Chance”

A sensitivity analysis should not be a collection of methods tried after the primary result is known until one gives a preferred answer.

Instead, it should be connected to a known uncertainty:

• What if the missing-data assumption is too optimistic?
• What if the covariance structure is misspecified?
• What if treatment discontinuation changes the interpretation?
• What if a small number of observations drive the result?

Multiplicity and the Estimand

Multiplicity planning should be connected to the actual confirmatory objectives. For example, if two endpoints are both required for success, a conjunctive strategy may have different implications from a strategy in which either endpoint can establish success.

Similarly, hierarchical testing can prioritize one estimand over another:

Adaptive Designs and ICH E9

Adaptive designs can modify specified aspects of a trial based on accumulating information while preserving the validity and integrity of the study.

Examples include:

• Sample-size re-estimation
• Arm dropping
• Response-adaptive randomization
• Population enrichment
• Seamless phase designs

Adaptive methods require careful planning of statistical operating characteristics, decision rules, information flow, and type I error control where applicable.

Clinical Interpretation of the Estimate

The final treatment-effect estimate should be interpreted in the context of:

• The target estimand
• The analysis population
• The endpoint definition
• The uncertainty around the estimate
• Intercurrent events
• Missing-data assumptions
• Multiplicity
• Protocol deviations
• Safety findings
• Clinical relevance

A numerical result without this context can be misleading.

A Worked Example: Week 24 Change From Baseline

Suppose a randomized trial has 200 participants, 100 per arm. The primary endpoint is change from baseline in a disease score at Week 24.

The correct interpretation is not simply “p=0.006, therefore the drug works.” A more complete interpretation is that the prespecified estimator produced an estimated treatment difference of −4.8 points, with a 95% confidence interval from −8.2 to −1.4, under the specified estimand and model assumptions.

Estimand-Based Interpretation of the Example

Suppose the estimand uses a treatment-policy strategy for rescue medication. Then the estimate is interpreted as the difference in Week 24 disease score that would be expected under assignment to Drug A versus placebo, with outcomes after rescue considered according to the treatment-policy definition.

If instead the estimand were hypothetical and asked what would have happened in the absence of rescue medication, the same numerical analysis result could not automatically be interpreted as the same clinical quantity.

What a Good SAP Should Make Clear

• Primary objective
• Primary estimand
• Primary endpoint
• Analysis population
• Intercurrent-event strategy
• Primary estimator
• Covariates
• Missing-data assumptions and methods
• Multiplicity control
• Interim analyses
• Sensitivity analyses
• Subgroup analyses
• Rules for deviations and exclusions
• Presentation of effect estimates and confidence intervals

Clinical Trial Statistical Review Checklist

Common Mistakes

1. Starting with the statistical test instead of the clinical question. Choosing “ANCOVA,” “logistic regression,” or “Cox regression” before defining the target treatment effect reverses the logical order.
2. Treating the estimand as another name for the endpoint. The endpoint is one attribute of an estimand; the estimand also addresses population, treatments, intercurrent events, and summary measure.
3. Ignoring intercurrent events. Discontinuation, rescue therapy, and treatment switching can materially change the treatment question.
4. Using an analysis population as an after-the-fact filter. Exclusions should be based on prespecified rules rather than favorable outcomes.
5. Reporting only p-values. Treatment-effect magnitude and precision are essential.
6. Assuming missing data are harmless because the percentage is small. A small amount of informative missingness can still matter.
7. Running many subgroup tests without a prespecified strategy. Apparent subgroup differences can arise by chance.
8. Changing the primary analysis after seeing unblinded results. Changes should follow appropriate governance and documentation.
9. Confusing a sensitivity analysis with an alternative primary analysis. The primary analysis should remain clearly identified.
10. Overinterpreting a statistically significant result. Statistical significance does not establish clinical importance.

How ICH E9 Changes the Way a Statistician Thinks

The most important shift is from asking:

to asking:

Once that question is answered, the design, endpoint, handling of intercurrent events, estimator, and sensitivity analyses can be aligned around it.

From E9 Principles to Production Programming

For a clinical-trial statistical programmer, E9 principles translate into concrete programming requirements.

Validation and QC

Statistical programming validation should test both the numerical result and the underlying logic.

• Confirm treatment assignments.
• Confirm analysis population flags.
• Recalculate key derived variables independently.
• Validate baseline selection.
• Validate endpoint windows.
• Validate handling of intercurrent events.
• Reproduce primary model results independently where feasible.
• Compare confidence intervals and p-values.
• Check that denominators match the SAP.
• Review extreme and influential observations.
• Confirm that TLF footnotes accurately describe the analysis.

Reporting the Primary Result

A strong clinical-study report should make it possible for a reviewer to answer four questions:

Quick Reference: The E9 Vocabulary

The Most Important Principles to Remember

Bottom Line

ICH E9 provides a framework for producing credible statistical evidence from clinical trials. ICH E9(R1) makes that framework more explicit by requiring investigators to define the treatment effect of interest through an estimand and to address intercurrent events as part of the clinical question rather than treating them only as missing-data problems.

The practical sequence is:

When these pieces are aligned, the resulting analysis is easier to interpret, defend, program, validate, and communicate. That alignment is the central statistical lesson of ICH E9.

References

International Council for Harmonisation (ICH). E9 Statistical Principles for Clinical Trials.
International Council for Harmonisation (ICH). E9(R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials. Step 4, 2019.
ICH. Efficacy Guidelines. E9 is listed among the ICH efficacy guidelines.
ICH. Guideline Implementation. Information on implementation of ICH guidelines by member regulatory authorities.