Sample Size for Proportions (Two-Sample)

Non-inferiority sample-size calculations require careful attention to the direction of the endpoint, the margin, one-sided type I error, and the exact analysis method.

Power Versus Sample Size

Increasing the sample size generally increases power.

For example, suppose all other assumptions remain fixed.

The relationship is not linear.

Moving from 80% to 90% power can require a meaningful increase in sample size, particularly when the target effect is modest.

Why 90% Power Is Common in Some Clinical Trials

An 80% power target corresponds to a 20% probability of failing to detect the specified effect under the planning alternative.

A 90% power target reduces this probability to 10%.

versus:

The appropriate target depends on the importance of the endpoint, regulatory expectations, consequences of a false-negative conclusion, feasibility, and other design considerations.

Sample Size and Multiplicity

If the trial has multiple primary hypotheses or multiple co-primary endpoints, the effective type I error allocation may differ from a simple 0.05 test.

For example, if a design allocates alpha across multiple hypotheses, the critical value entering the sample-size calculation may be different.

Clustered or Correlated Binary Outcomes

The standard two-sample proportion formula assumes independent observations.

That assumption may not hold in cluster-randomized trials, repeated binary measurements, or other correlated designs.

Correlation reduces the effective information contributed by each observation.

In a cluster-randomized study, a design effect may be applied, often involving the intracluster correlation coefficient (ICC) and cluster size.

where:

• \(m\) = average cluster size
• \(\rho\) = intracluster correlation coefficient

The resulting sample size can be substantially larger than the independent two-sample calculation.

Stratification and Covariate Adjustment

Randomized trials often use stratification or covariate-adjusted analyses.

The simple two-proportion calculation provides a useful baseline, but the actual analysis may be based on:

• Cochran–Mantel–Haenszel methods
• Logistic regression
• Stratified risk differences
• Adjusted risk ratios
• Other prespecified covariate-adjusted models

If the primary analysis differs materially from the simple unadjusted two-proportion comparison, the sample-size method should be evaluated to ensure that it provides adequate power for the intended analysis.

R Implementation

R provides several ways to perform two-sample proportion sample-size calculations.

A standard starting point is the built-in power.prop.test() function.

power.prop.test(
  p1 = 0.60,
  p2 = 0.40,
  power = 0.80,
  sig.level = 0.05,
  alternative = "two.sided"
)

This function returns the required approximate sample size per group under its specified method.

Solving for Sample Size Directly

The same function can be used without specifying the sample size.

result <- power.prop.test(
  p1 = 0.60,
  p2 = 0.40,
  power = 0.80,
  sig.level = 0.05,
  alternative = "two.sided"
)

result

The output includes the calculated sample size along with the other planning parameters.

Checking the Result

A good practice is to verify the result using a second calculation.

p1 <- 0.60
p2 <- 0.40
alpha <- 0.05
power_target <- 0.80

z_alpha <- qnorm(1 - alpha / 2)
z_beta <- qnorm(power_target)

p_bar <- (p1 + p2) / 2

n <-
  2 *
  (
    z_alpha * sqrt(2 * p_bar * (1 - p_bar)) +
    z_beta * sqrt(
      p1 * (1 - p1) +
      p2 * (1 - p2)
    )
  )^2 /
  (p1 - p2)^2

ceiling(n)

This reproduces the approximate normal-theory calculation described earlier.

Creating a Reusable R Function

For repeated sample-size calculations, it is useful to create a function.

two_prop_n <- function(
  p1,
  p2,
  alpha = 0.05,
  power = 0.80
) {

  z_alpha <- qnorm(1 - alpha / 2)
  z_beta <- qnorm(power)

  p_bar <- (p1 + p2) / 2

  n <-
    2 *
    (
      z_alpha * sqrt(2 * p_bar * (1 - p_bar)) +
      z_beta * sqrt(
        p1 * (1 - p1) +
        p2 * (1 - p2)
      )
    )^2 /
    (p1 - p2)^2

  ceiling(n)
}

two_prop_n(
  p1 = 0.60,
  p2 = 0.40,
  alpha = 0.05,
  power = 0.80
)

Exploring Different Effect Sizes in R

One of the most useful applications is to see how sample size changes as the target effect changes.

control <- 0.40

experimental_rates <-
  c(0.45, 0.50, 0.55, 0.60, 0.65, 0.70)

sample_sizes <- sapply(
  experimental_rates,
  function(p1) {
    two_prop_n(
      p1 = p1,
      p2 = control,
      alpha = 0.05,
      power = 0.80
    )
  }
)

data.frame(
  experimental_rate = experimental_rates,
  control_rate = control,
  difference = experimental_rates - control,
  n_per_group = sample_sizes,
  total_n = 2 * sample_sizes
)

This table demonstrates an important planning concept: small changes in the target effect size can produce large changes in the required sample size.

Exploring Power in R

After selecting a sample size, investigators can calculate the resulting power for different true treatment effects.

p1_values <- c(
  0.45,
  0.50,
  0.55,
  0.60,
  0.65,
  0.70
)

power_values <- sapply(
  p1_values,
  function(p1) {

    power.prop.test(
      p1 = p1,
      p2 = 0.40,
      n = 194,
      sig.level = 0.05,
      alternative = "two.sided"
    )$power
  }
)

data.frame(
  p1 = p1_values,
  p2 = 0.40,
  power = power_values
)

This is useful because the study's actual power is not a single universal number. It depends on the true underlying proportions.

Power as a Function of the True Difference

Define:

The resulting power surface depends on both proportions.

For a fixed control proportion, power generally increases as the experimental proportion moves farther away from the control value.

Sample Size Sensitivity Analysis

A strong statistical analysis plan should not rely on only one arbitrary planning scenario.

Investigators can evaluate a range of assumptions.

Such sensitivity analyses help the study team understand how robust the enrollment requirement is to uncertainty in the assumptions.

Choosing the Planning Proportions

Sources of information can include:

• Previous randomized clinical trials
• Historical control data
• Phase I or Phase II studies
• Published literature
• Real-world data
• Pilot studies
• Clinical expert judgment
• Regulatory or development-program objectives

Historical data should be interpreted carefully because response rates can change across populations, endpoint definitions, treatment lines, follow-up periods, and assessment criteria.

The Control Rate Is Especially Important

Suppose the experimental treatment is expected to have a 60% response rate.

If the control rate is assumed to be 30%, the expected difference is:

But if the control rate is expected to be 50%, the difference is only:

These scenarios can require dramatically different sample sizes.

Unequal Group Costs

Statistical efficiency is not the only consideration when choosing an allocation ratio.

Suppose one treatment is substantially more expensive than another. An investigator may prefer unequal allocation even though it requires more total patients.

The design therefore involves a tradeoff between:

• Statistical efficiency
• Treatment exposure
• Cost
• Safety information
• Recruitment feasibility
• Ethical considerations

Ethical Considerations

Sample-size planning is also an ethical issue.

An underpowered study can expose participants to an experimental intervention without a reasonable probability of answering the scientific question.

An unnecessarily oversized study can expose more participants than necessary.

Interim Analyses and Sample Size Re-Estimation

Some trials allow sample-size re-estimation during the study.

For example, investigators may update an estimate of the nuisance parameter or reassess assumptions about event rates.

However, unplanned changes based on observed treatment differences can affect type I error and should not be treated as a simple administrative adjustment.

If adaptive sample-size re-estimation is planned, its statistical properties should be established in advance.

Missing Data

The standard formula assumes that the analyzed observations contribute valid binary outcomes.

Missing primary endpoint data can reduce the effective sample size.

The protocol and statistical analysis plan should therefore define:

• What constitutes an evaluable patient
• How missing responses are handled
• Whether replacement is permitted
• How intercurrent events affect the endpoint
• Which analysis population is primary

The dropout inflation should be consistent with the assumptions used in the analysis.

Intention-to-Treat Versus Per-Protocol

In randomized trials, binary endpoints may be analyzed using different populations.

An intention-to-treat analysis generally includes patients according to their randomized treatment assignment.

A per-protocol analysis may exclude certain protocol deviations.

If the primary analysis is based on intention-to-treat, the sample-size calculation should be interpreted in terms of the expected endpoint availability in that population.

Common Mistakes

1. Using the wrong effect size. Sample size should be based on the smallest clinically meaningful difference, not simply the largest anticipated difference.
2. Confusing total sample size with sample size per group. Always verify what the formula or software reports.
3. Forgetting dropout inflation. The statistical calculation often describes evaluable patients rather than patients who must be randomized.
4. Using a two-sided formula for a one-sided design or vice versa. The critical value must match the planned hypothesis test.
5. Choosing a one-sided test merely to obtain a smaller sample size. The testing direction must be scientifically justified.
6. Ignoring unequal allocation. The allocation ratio affects statistical efficiency.
7. Assuming the normal approximation is always appropriate. Rare events and proportions near 0 or 1 may require more careful methods.
8. Ignoring multiplicity. Multiple primary hypotheses may require an adjusted alpha allocation.
9. Using unrealistic response-rate assumptions. Poor assumptions can result in a study that is underpowered or unnecessarily large.
10. Reporting only the final sample size. The statistical method, assumptions, alpha, power, allocation, and dropout assumptions should also be documented.

A Practical Two-Sample Proportion Workflow

What Should Be Reported in the Protocol?

A reproducible sample-size section should specify at least:

• Primary endpoint
• Statistical hypothesis
• Expected proportion in each group
• Clinically meaningful difference
• Type I error
• One-sided or two-sided testing
• Target power
• Allocation ratio
• Sample-size calculation method
• Continuity correction, if applicable
• Required evaluable sample size
• Expected dropout rate
• Total enrollment target
• Any adjustment for multiplicity
• Any inflation for clustering or other design effects

Worked Example Summary

The main example can be summarized as follows.

The Most Important Concept

The most important idea in two-sample proportion sample-size planning is that sample size is driven by the signal you need to distinguish from random variation.

The signal is the difference:

The statistical noise depends on the proportions themselves, the allocation ratio, and the sample size.

The study must be large enough that the anticipated clinically meaningful difference can be detected with the desired probability while maintaining the specified type I error.

In practical terms:

• Larger treatment differences require fewer patients.
• Smaller treatment differences require more patients.
• Higher power requires more patients.
• Smaller alpha requires more patients.
• Unequal allocation generally requires more total patients.
• Dropout increases the enrollment target.
• Rare events require particular care with asymptotic methods.

References

Fleiss, J.L., Tytun, A. & Ury, H.K. (1980). A simple approximation for calculating sample sizes for comparing independent proportions. Biometrics, 36(2), 343–346.
Fleiss, J.L., Levin, B. & Paik, M.C. (2003). Statistical Methods for Rates and Proportions. 3rd ed. Wiley.
Chow, S.-C., Shao, J., Wang, H. & Lokhnygina, Y. (2017). Sample Size Calculations in Clinical Research. 3rd ed. CRC Press.
Julious, S.A. (2009). Sample Sizes for Clinical Trials. Chapman & Hall/CRC.
Agresti, A. (2013). Categorical Data Analysis. 3rd ed. Wiley.
Newcombe, R.G. (1998). Interval estimation for the difference between independent proportions: comparison of eleven methods. Statistics in Medicine, 17, 873–890.