Sample Size for Non-Inferiority Trials

Introduction

A non-inferiority trial is designed to determine whether a new treatment is not unacceptably worse than an established active control. The new treatment does not necessarily need to be statistically superior. Instead, the objective is to demonstrate that any loss in efficacy is smaller than a prespecified and clinically acceptable amount.

Non-inferiority designs are common when the experimental treatment may offer advantages that are not captured by the primary efficacy endpoint. For example, a new treatment might be easier to administer, safer, less expensive, more convenient, or associated with fewer adverse effects while retaining sufficient efficacy.

The sample size calculation is therefore fundamentally different from an ordinary superiority calculation. The critical quantity is not simply the expected treatment effect. It is the relationship between the expected treatment difference and the non-inferiority margin.

Superiority vs. Non-Inferiority

Consider a randomized trial comparing an experimental treatment \(T\) with an active control \(C\). Let the treatment effect be defined as:

for a continuous endpoint where larger values are better.

A superiority trial might test:

or, for a directional superiority hypothesis:

A non-inferiority trial instead specifies a negative margin \(\Delta>0\) representing the largest clinically acceptable loss:

The null hypothesis says that the experimental treatment is inferior by at least \(\Delta\). The alternative says that the treatment is sufficiently close to the control that an inferiority of \(\Delta\) or greater can be ruled out.

The Non-Inferiority Margin

The most important design parameter in a non-inferiority trial is often the non-inferiority margin.

The margin, commonly denoted by \(\Delta\), defines the largest loss of treatment effect that can still be considered clinically acceptable.

For a continuous endpoint where higher values are better:

For example, if:

then a treatment difference greater than \(-5\) is considered compatible with non-inferiority.

What Makes a Margin Difficult to Choose?

Suppose the active control historically improves a clinical outcome by approximately 10 units compared with placebo. An investigator might decide that preserving at least 50% of that established effect is necessary.

That could motivate a non-inferiority margin of:

The statistical calculation can then determine how many patients are needed to estimate the treatment difference precisely enough to demonstrate non-inferiority.

But the statistical calculation itself does not determine whether 5 units is clinically appropriate.

One-Sided Type I Error

Non-inferiority trials generally use a one-sided type I error because the scientific question is directional.

The concern is whether the experimental treatment is unacceptably worse than the control.

A common choice is:

The corresponding standard normal critical value is approximately:

Power

Power is the probability of successfully demonstrating non-inferiority when the assumed true treatment effect is consistent with the planning alternative.

If the assumed true difference is \(\delta_A\), then:

Common target values include:

• 80% power
• 85% power
• 90% power
• 95% power for especially important confirmatory settings

The Key Relationship Between Effect and Margin

Suppose the expected true treatment difference is:

and the non-inferiority margin is:

The distance between the expected effect and the non-inferiority boundary is:

This distance determines how easy or difficult it is to demonstrate non-inferiority.

If the expected treatment effect were instead \(-3\), the distance would be:

A much smaller distance means that substantially more information is needed.

Sample Size for a Continuous Endpoint

Consider a parallel-group trial with equal allocation and a continuous primary endpoint. Assume:

• Common standard deviation \(\sigma\)
• Expected treatment difference \(\delta_A\)
• Non-inferiority margin \(\Delta\)
• One-sided type I error \(\alpha\)
• Power \(1-\beta\)

For an approximate normal-theory calculation, the required sample size per treatment group is:

The denominator contains the distance between the assumed treatment effect and the non-inferiority boundary.

Because:

the quantity \(\delta_A+\Delta\) is the relevant effect-to-margin distance when the superiority direction is positive.

Worked Example: Continuous Endpoint

Suppose a randomized non-inferiority trial compares a new treatment with an established active control using a continuous clinical endpoint.

Step 1: Identify the Non-Inferiority Boundary

The hypothesis is:

The planned true treatment difference is 0:

Therefore, the distance between the assumed effect and the boundary is:

Step 2: Obtain the Critical Values

For one-sided \(\alpha=0.025\):

For 90% power:

Step 3: Calculate the Sample Size

Substitute the values into the continuous-endpoint formula:

The result is approximately:

Because sample size must be an integer:

Therefore the required randomized sample size before dropout inflation is:

Step 4: Inflate for Dropout

Suppose the anticipated dropout rate is 10%. The retention rate is:

The inflation formula is:

Therefore:

Rounding upward gives:

Thus, a practical design would randomize 192 patients per treatment group.

The Confidence-Interval Interpretation

A non-inferiority conclusion can also be expressed through a confidence interval.

Suppose the treatment effect is defined as:

The trial demonstrates non-inferiority when the lower confidence limit is above the non-inferiority boundary:

For a one-sided 2.5% test, this is commonly equivalent to examining the lower bound of a 95% two-sided confidence interval.

Sample Size and Confidence-Interval Precision

The reason non-inferiority trials can require large sample sizes is easy to see from the confidence interval.

A narrow confidence interval requires a large amount of information.

If the non-inferiority margin is narrow, the confidence interval must be even more precise to remain entirely above that boundary.

For example, changing the margin from 5 units to 3 units changes the effect distance from:

to:

Since sample size is proportional to the inverse square of this distance:

the required sample size can increase dramatically.

Effect of the Non-Inferiority Margin

This illustrates why a scientifically justified margin is so important. A smaller margin is not automatically "better" if it is not supported by the clinical context, and it can have major consequences for trial feasibility.

Binary Endpoints

Many non-inferiority trials use binary endpoints such as:

• Clinical response
• Remission
• Event-free status
• Composite success/failure
• Occurrence of an adverse outcome

For a binary endpoint, let:

and:

A risk-difference treatment effect can be defined as:

For a higher-is-better endpoint, the non-inferiority hypothesis becomes:

Approximate Binary Sample Size Formula

For equal allocation, a commonly used normal-approximation framework can be expressed as:

where \(n\) is the approximate sample size per group, and the null-boundary treatment rate is:

This is an approximation. For confirmatory clinical trials with binary endpoints, investigators often use more specialized methods such as score-based approaches, including Farrington–Manning-type calculations, rather than relying on a simple Wald approximation.

Worked Example: Binary Endpoint

Suppose a randomized trial evaluates a new treatment against an active control using a binary success endpoint.

Step 1: Define the Hypotheses

The treatment difference is:

The non-inferiority margin is 10 percentage points:

Therefore:

The planned treatment difference is:

The distance between the assumed effect and the non-inferiority boundary is:

Step 2: Null-Boundary Event Rate

Under the null boundary:

Therefore:

This represents the treatment response rate at the non-inferiority boundary when the control rate is 80%.

Step 3: Approximate Sample Size

Using the normal approximation:

This gives approximately:

or approximately:

This is an illustrative normal-approximation calculation. A production clinical-trial calculation should generally be verified using the exact analysis method planned for the binary endpoint.

Why Binary Sample Size Can Be Large

At first glance, 660 patients may seem surprisingly large for a 10-percentage point non-inferiority margin.

The reason is that the trial must distinguish an expected difference of 0 from a clinically unacceptable difference of \(-0.10\), while maintaining a stringent one-sided type I error and high power.

Binary outcomes also contain less information per patient than a highly precise continuous measurement under some circumstances.

Different Effect Scales

Non-inferiority margins can be specified on different scales. Common choices include:

• Risk difference
• Risk ratio
• Odds ratio
• Difference in means
• Difference in transformed means
• Hazard ratio

The direction of the hypothesis depends on how the treatment effect is defined.

Hazard Ratios and Time-to-Event Endpoints

For time-to-event outcomes, the treatment effect may be expressed as a hazard ratio:

When lower hazard is better, a non-inferiority trial might define:

For example, a margin of:

would mean that hazard ratios at or above 1.25 are considered unacceptably inferior under this particular definition.

For time-to-event trials, sample size is often expressed in terms of the required number of events first, followed by the number of patients needed to generate those events.

Event-Based Sample Size

A useful conceptual distinction is:

This is one reason that a simple two-group continuous-endpoint formula should not be applied indiscriminately to time-to-event non-inferiority trials.

Allocation Ratio

The examples above assume equal allocation:

Equal allocation is often statistically efficient when the two groups have similar costs and variances.

However, non-equal allocation can be useful when:

• The control treatment is scarce or expensive.
• The experimental treatment is easier to recruit.
• Safety information requires more exposure to one treatment.
• There are ethical or operational reasons for unequal allocation.

Unequal allocation changes the variance and therefore changes the required sample size.

Sample Size With Unequal Allocation

For a continuous endpoint with allocation ratio:

the variance of the treatment difference becomes:

The design should therefore solve for the treatment and control group sizes simultaneously rather than simply multiplying an equal-allocation result.

Dropout Inflation

Dropout is particularly important in non-inferiority trials because missing data can affect both the precision of the treatment-effect estimate and the interpretation of the trial.

If the required evaluable sample size is \(N_E\) and the anticipated dropout proportion is \(d\), then:

where \(N_R\) is the number to randomize.

For example, with 15% anticipated dropout:

The result should always be rounded upward.

Why Dropout Is Especially Important in Non-Inferiority Trials

In a superiority trial, missing outcomes can sometimes make the treatment groups appear more similar simply because information is lost.

In a non-inferiority trial, this can be particularly problematic because similarity between treatments is part of the desired conclusion.

Consequently, a high dropout rate can make it easier to observe apparent similarity while simultaneously weakening the evidence needed to establish non-inferiority.

Assay Sensitivity

A non-inferiority trial generally depends on the active control having the ability to demonstrate its expected effect under the conditions of the trial. This is often described as assay sensitivity.

If the active control performs poorly because of changes in population, adherence, endpoint assessment, treatment implementation, or other factors, the experimental treatment and control may appear similar even when both are less effective than expected.

This creates an important distinction:

This is one reason non-inferiority design requires more than simply selecting a sample size formula.

Preserving the Control Effect

The non-inferiority margin should generally be connected to the historical effect of the active control.

Conceptually, suppose historical evidence indicates that the control has an effect of \(M_1\) relative to placebo. The non-inferiority margin should preserve a clinically justified fraction of that effect.

For example, if the historical control effect is 10 units and investigators wish to preserve at least 50%:

The precise derivation depends on the endpoint, estimand, historical evidence, and regulatory framework.

Analysis Population Considerations

Non-inferiority trials commonly place substantial emphasis on both:

• The intention-to-treat population
• The per-protocol population

This is related to the concern that deviations from treatment, nonadherence, and protocol violations can make treatments appear artificially similar.

The exact analysis strategy should be prespecified.

Non-Inferiority Margin vs. Effect Size

It is useful to distinguish three quantities:

The sample size depends strongly on the third quantity.

For example:

gives an effect distance of 5.

But:

gives:

and therefore a much larger sample size.

Effect of Power

Increasing power increases sample size because the study must have greater ability to distinguish the assumed alternative from the non-inferiority boundary.

For the continuous endpoint:

For example, moving from 80% to 90% power increases \(z_{1-\beta}\) from approximately 0.842 to 1.282.

Effect of Alpha

Reducing alpha makes the statistical criterion more stringent and generally increases the required sample size.

For one-sided tests:

This is one reason confirmatory non-inferiority trials often require substantial sample sizes.

Effect of Variability

For a continuous endpoint:

Therefore, if the standard deviation doubles, the required sample size increases by approximately a factor of four, assuming all other parameters remain fixed.

This makes a reliable estimate of the standard deviation especially important during planning.

Sample Size Sensitivity Analysis

A robust sample size section should not rely on one optimistic assumption. Investigators should evaluate the effect of changing:

• Expected treatment effect
• Non-inferiority margin
• Standard deviation
• Control event rate
• Power
• Alpha
• Dropout rate
• Allocation ratio

For example, a continuous-endpoint sensitivity analysis might look like:

These values are illustrative and demonstrate the direction of sensitivity; the final design should use the exact calculation method specified in the protocol.

R Implementation: Continuous Endpoint

The basic continuous-endpoint calculation can be implemented directly in R.

alpha <- 0.025
power <- 0.90

sigma <- 10
delta <- 0
margin <- 5

z_alpha <- qnorm(1 - alpha)
z_beta  <- qnorm(power)

n_per_group <-
  2 * sigma^2 *
  (z_alpha + z_beta)^2 /
  (delta + margin)^2

ceiling(n_per_group)

The result is:

# 172

Inflating the R Calculation for Dropout

dropout <- 0.10

n_randomized_per_group <-
  ceiling(
    n_per_group / (1 - dropout)
  )

n_randomized_per_group

This gives:

# 192

Thus the planned randomized sample size is:

A Reusable R Function

ni_sample_size_continuous <- function(
  sigma,
  delta,
  margin,
  alpha = 0.025,
  power = 0.90,
  dropout = 0
) {

  z_alpha <- qnorm(1 - alpha)
  z_beta  <- qnorm(power)

  n_evaluable <-
    2 * sigma^2 *
    (z_alpha + z_beta)^2 /
    (delta + margin)^2

  n_randomized <-
    ceiling(
      n_evaluable / (1 - dropout)
    )

  list(
    evaluable_per_group =
      ceiling(n_evaluable),
    evaluable_total =
      2 * ceiling(n_evaluable),
    randomized_total =
      2 * n_randomized
  )
}

ni_sample_size_continuous(
  sigma = 10,
  delta = 0,
  margin = 5,
  alpha = 0.025,
  power = 0.90,
  dropout = 0.10
)

Checking the Assumptions

The sample size is only as reliable as its planning assumptions. For a continuous endpoint, investigators should examine whether:

• The assumed standard deviation is realistic.
• The expected treatment difference is clinically plausible.
• The endpoint has approximately the assumed distribution.
• The allocation ratio is correct.
• The dropout assumption is realistic.
• The non-inferiority margin is clinically justified.

Historical studies, pilot data, previous trials, and blinded sample-size re-estimation strategies may be relevant depending on the protocol.

Sample Size Re-Estimation

In some settings, the assumed variance may be uncertain. A blinded sample-size review can sometimes be considered to evaluate whether the observed variability is consistent with the planning assumption.

Any such procedure must be designed carefully so that it does not compromise the trial's type I error or introduce operational bias.

What Happens If the Expected Effect Is Negative?

Suppose the experimental treatment is expected to be slightly worse than the control, with:

and:

The distance from the non-inferiority boundary is:

The sample size therefore increases relative to the scenario where the expected treatment difference is 0.

This is an important planning principle: the closer the anticipated treatment effect is to the margin, the larger the sample size becomes.

What If the Experimental Treatment Is Expected to Be Better?

Suppose:

with the same margin:

Then:

The expected effect is farther from the non-inferiority boundary, so fewer patients are required for the same alpha and power under the simple normal-theory calculation.

Non-Inferiority Is Not Equivalence

These concepts are closely related but not identical.

For equivalence, the hypotheses are typically:

versus:

Equivalence therefore requires ruling out both clinically meaningful benefit and clinically meaningful harm.

Non-Inferiority Does Not Mean "No Difference"

This is one of the most important concepts in the entire topic.

Suppose the 95% confidence interval for the treatment difference is:

and the non-inferiority margin is:

Because:

the confidence interval does not include the unacceptable inferiority region. Non-inferiority can therefore be demonstrated even though the confidence interval includes zero.

Common Sample Size Mistakes

1. Using a superiority formula without changing the hypothesis. Non-inferiority calculations must incorporate the non-inferiority margin.
2. Choosing the margin after calculating sample size. The margin should be clinically justified before the statistical calculation.
3. Using two-sided alpha automatically. The planned one-sided type I error must be clearly specified.
4. Confusing the margin with the expected treatment effect. They are separate design quantities.
5. Ignoring the direction of the endpoint. The sign of the treatment effect and margin depends on how the endpoint is defined.
6. Ignoring dropout. The number required for analysis is not necessarily the number that must be randomized.
7. Using a simple Wald formula for every binary endpoint. More appropriate score-based or exact methods may be needed.
8. Ignoring the control event-rate assumption. Binary endpoint sample size can change substantially when the expected event rates change.
9. Assuming non-inferiority is equivalent to no significant difference. Failure to reject superiority does not establish non-inferiority.
10. Failing to consider assay sensitivity. Poor performance of the active control can undermine the interpretation.

A Practical Sample Size Workflow

What Should Be Reported in the Protocol?

A reproducible non-inferiority sample size section should include:

• Primary endpoint
• Definition and direction of the treatment effect
• Non-inferiority margin
• Clinical justification for the margin
• Expected treatment effect
• Control mean, standard deviation, or event rate as applicable
• One-sided type I error
• Target power
• Allocation ratio
• Statistical method used for the sample size calculation
• Expected dropout or non-evaluable rate
• Required evaluable sample size
• Number to randomize
• Any inflation for interim analyses or other design features
• Sensitivity analyses

Worked Example Summary

The Most Important Concept

The most important idea in non-inferiority sample size planning is that the trial is designed around the distance between the expected treatment effect and the non-inferiority boundary.

For the continuous endpoint:

Every major design parameter has an intuitive interpretation:

• Larger variability → larger sample size.
• Smaller non-inferiority margin → larger sample size.
• Expected effect closer to the margin → larger sample size.
• Higher power → larger sample size.
• Smaller alpha → larger sample size.
• More dropout → more patients randomized.

References

ICH E9. Statistical Principles for Clinical Trials. International Council for Harmonisation.

ICH E9(R1). Addendum on Estimands and Sensitivity Analysis in Clinical Trials. International Council for Harmonisation.

ICH E10. Choice of Control Group and Related Issues in Clinical Trials. International Council for Harmonisation.

Piaggio, G., Elbourne, D.R., Altman, D.G., Pocock, S.J. & Evans, S.J.W. (2006). Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT statement. JAMA, 295(10), 1152–1160.

Schuirmann, D.J. (1987). A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15, 657–680.

Blackwelder, W.C. (1982). Proving the null hypothesis in clinical trials. Controlled Clinical Trials, 3(4), 345–353.

Farrington, C.P. & Manning, G. (1990). Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference. Statistics in Medicine, 9(12), 1447–1454.