Sample Size for Two-Sample t-Tests: A Worked Example

Introduction

Many clinical trials compare the mean value of a continuous endpoint between two independent treatment groups.

Examples include change from baseline in:

• Blood pressure
• Cholesterol
• Biomarker concentrations
• Pain scores
• Lung function
• Quality-of-life measurements
• Other continuous clinical or laboratory endpoints

When the primary analysis compares two independent means, a two-sample t-test is one of the standard statistical methods for testing whether the population means differ.

Before the study begins, however, investigators must determine how many patients are required.

The sample size depends primarily on four statistical quantities:

The Two-Sample t-Test

Suppose there are two independent groups:

• Group 1 has population mean \(\mu_1\)
• Group 2 has population mean \(\mu_2\)

The parameter of interest is usually the difference:

For a two-sided test, the hypotheses are:

For a one-sided test in which Group 1 is expected to have a larger mean:

The sample size calculation is based on the difference that the investigators consider clinically meaningful, not necessarily the smallest difference that could be statistically significant.

What Does "Clinically Meaningful Difference" Mean?

Let:

represent the absolute difference that the study should be powered to detect.

For example, suppose a treatment is expected to reduce systolic blood pressure by 5 mmHg more than control.

Then:

The sample size calculation asks: How many patients are required to have adequate power to detect a 5 mmHg difference?

The Role of Standard Deviation

The second major input is the expected standard deviation.

Let:

denote the assumed common standard deviation when equal variances are used for planning.

If:

and the clinically meaningful difference is:

then the signal-to-noise ratio is:

This standardized difference is commonly called Cohen's \(d\):

A larger \(d\) means the treatment difference is large relative to the background variability.

Cohen's d

Cohen's standardized effect size is:

For equal-variance planning, a commonly used rough interpretation is:

These labels are only general reference points. In a clinical trial, the clinically meaningful difference should be defined from subject-matter considerations rather than from Cohen's labels.

The Basic Sample Size Formula

For two independent groups with equal sample sizes and a common planning standard deviation, a widely used normal-approximation formula is:

where \(n\) is the required sample size per group.

Here:

Why the Formula Contains a Factor of 2

For two independent groups with equal sample sizes:

The difference between two independent means therefore has twice the per-group variance contribution.

This leads to the factor of 2 in the equal-allocation sample size formula.

Two-Sided Versus One-Sided Tests

The choice of sidedness affects sample size.

For a two-sided test with significance level \(\alpha\):

is used.

For a one-sided test:

is used instead.

Because the one-sided critical value is smaller, a one-sided design generally requires fewer patients for the same power and effect size.

Power and Type II Error

Power is:

Common target power values include:

• 80%
• 85%
• 90%
• 95%

For example, 80% power means that if the true treatment difference equals the planned effect, the study has approximately an 80% probability of producing a statistically significant result under the specified analysis assumptions.

The corresponding type II error is:

Thus, 80% power corresponds to:

How Sample Size Changes With Power

Higher power requires more patients.

For a two-sided test with \(\alpha=0.05\):

The relationship is not linear. Because the relevant quantity is the square of the sum of the two critical values, moving from 80% to 90% power can require a substantial increase in sample size.

A Complete Worked Example

Suppose a randomized clinical trial compares a new treatment with placebo using a continuous primary endpoint.

The investigators expect:

• A clinically meaningful treatment difference of 5 units
• A common standard deviation of 10 units
• A two-sided type I error of 5%
• 80% power
• Equal allocation to the two groups

Step 1: Calculate the Standardized Effect Size

The standardized effect is:

Substituting the planning values:

Thus, the study is designed to detect a moderate standardized difference of 0.50.

Step 2: Identify the Critical Values

For a two-sided test with:

the standard normal critical value is:

For 80% power:

Step 3: Apply the Formula

The equal-allocation formula is:

Substitute:

Therefore:

which gives approximately:

Because sample size must be an integer, round up:

Therefore, the total analyzable sample size is:

Step 4: Verify the Effect Size

The calculation can also be expressed entirely in terms of Cohen's \(d\):

For \(d=0.50\):

Again, this rounds to:

Why We Round Up

Suppose the calculation produces:

The study cannot enroll 63.01 patients.

Rounding down to 63 would produce slightly less power than the calculated target.

Therefore, sample size calculations should generally be rounded upward:

where \(\lceil\cdot\rceil\) denotes the ceiling function.

Analyzable Sample Size vs. Enrollment Sample Size

The 128 patients calculated above are the number of patients expected to provide usable primary endpoint information.

Real trials may experience:

• Dropout
• Withdrawal
• Loss to follow-up
• Missing endpoint measurements
• Protocol deviations affecting analysis eligibility
• Other forms of non-evaluability

If 10% of randomized patients are expected to be lost or otherwise non-evaluable, simply adding 10% is not mathematically correct.

Instead, divide by the expected retention proportion.

Inflating for 10% Attrition

Suppose the expected attrition rate is:

The expected retention rate is:

Therefore:

For the worked example:

Rounding up gives:

With equal allocation, this means:

Common Attrition Adjustments

How Effect Size Drives Sample Size

From the sample size formula:

when the standard deviation and other design parameters are fixed.

This squared relationship is extremely important.

If the planned effect is cut in half, the required sample size approximately quadruples.

*Values are approximate normal-approximation results for a two-sided \(\alpha=0.05\) test and 80% power and may differ slightly from exact noncentral-t calculations.

How Variability Drives Sample Size

The sample size formula contains:

Therefore, doubling the standard deviation approximately quadruples the required sample size if the clinically meaningful difference remains fixed.

This is why reliable estimates of endpoint variability are so important during study planning.

Sources of the Standard Deviation

The planning standard deviation may come from:

• Previous randomized trials
• Pilot studies
• Published literature
• Internal historical data
• Earlier phases of the development program
• External databases or registries

The source should be scientifically defensible and reasonably comparable to the planned study population, endpoint definition, and analysis time point.

Sample Size Sensitivity Analysis

Suppose the planned difference remains 5 units but the standard deviation is uncertain.

This illustrates why the variability assumption deserves as much attention as the treatment effect assumption.

Equal Allocation

The standard formula assumes equal allocation:

For example:

Equal allocation is statistically efficient when the per-patient cost and variance are approximately comparable between groups.

It is also operationally simple and is common in randomized clinical trials.

Unequal Allocation

Sometimes investigators deliberately use an allocation ratio other than 1:1.

Suppose:

where \(k\) is the ratio of the second group's sample size to the first group's sample size.

For two independent groups with common variance, the variance of the difference is:

If \(n_2=kn_1\), then:

The corresponding normal-approximation sample size for Group 1 is:

and:

Why Equal Allocation Is Usually Efficient

For a fixed total sample size, equal allocation minimizes:

when the two groups have the same variance and per-patient information.

Thus, a 1:1 allocation generally provides the greatest power for a fixed total number of patients.

When the Two Groups Have Different Variances

The equal-variance calculation assumes:

In practice, the two groups may have different variances.

A Welch two-sample t-test does not require the equal-variance assumption.

For sample size planning, investigators can use expected values \(\sigma_1\) and \(\sigma_2\) and calculate the variance of the treatment difference as:

The general principle remains the same: sample size must be large enough that the planned treatment difference is distinguishable from the expected sampling variability.

Two-Sample t-Test vs. Paired t-Test

A common source of sample size errors is confusing an independent two-sample design with a paired design.

A two-sample t-test is appropriate when the observations in the two groups are independent.

Examples include:

• Drug versus placebo in different patients
• Treatment A versus Treatment B in different patients
• Two independent patient cohorts

A paired t-test is appropriate when observations are naturally paired.

Examples include:

• Before versus after measurements on the same patient
• Matched subjects
• Repeated measurements that have been reduced to within-subject differences

Power Is Based on the True Effect

Suppose the study is powered for:

and the true difference is actually only 3 units.

The study does not still have 80% power for the 3-unit effect.

The power calculation is tied to the effect size used for planning.

In general:

and power changes as the true effect changes.

Power Across Possible Effect Sizes

For the worked example, the study is designed around a standardized effect of \(d=0.50\).

A smaller true effect will generally have lower power, while a larger true effect will generally have higher power.

This is why a power calculation should always state the effect size for which power is being evaluated.

The t Distribution and Exact Sample Size

The simple formula uses standard normal quantiles.

A more exact calculation uses the noncentral t distribution because the two-sample t-test uses a t statistic and estimates the variance from the data.

For equal group sizes, the degrees of freedom under the conventional pooled two-sample t-test are:

The exact power is determined by the noncentrality parameter:

For equal allocation:

The exact calculation evaluates the probability that the noncentral t statistic falls beyond the appropriate critical value.

R Implementation With pwr

The R package pwr provides a convenient function for two-sample t-test power calculations.

install.packages("pwr")
library(pwr)

pwr.t.test(
  d = 0.50,
  sig.level = 0.05,
  power = 0.80,
  type = "two.sample",
  alternative = "two.sided"
)

The key inputs are:

• d = 0.50: standardized effect size
• sig.level = 0.05: type I error
• power = 0.80: desired power
• type = "two.sample": independent two-sample test
• alternative = "two.sided": two-sided hypothesis

The resulting continuous sample-size estimate is approximately:

n = 63.76576

Thus:

R Implementation Using the Planning Formula

The normal-approximation calculation can also be written directly in R.

alpha <- 0.05
power <- 0.80

delta <- 5
sigma <- 10

z_alpha <- qnorm(1 - alpha / 2)
z_beta  <- qnorm(power)

n_per_group <-
  2 * sigma^2 *
  (z_alpha + z_beta)^2 /
  delta^2

ceiling(n_per_group)

The result is:

# 64

Therefore the total analyzable sample size is:

2 * ceiling(n_per_group)

# 128

Exact Power for 64 Patients Per Group

The exact power can be evaluated using the noncentral t distribution.

n <- 64
delta <- 5
sigma <- 10
alpha <- 0.05

df <- 2 * n - 2

lambda <-
  delta /
  (sigma * sqrt(2 / n))

tcrit <- qt(1 - alpha / 2, df)

power_exact <-
  pt(-tcrit, df, ncp = lambda) +
  pt(tcrit, df, ncp = lambda, lower.tail = FALSE)

power_exact

The resulting exact power is approximately 80%, with the precise value depending on the exact implementation and numerical calculation.

Calculating Sample Size for 90% Power

Suppose the investigators decide that 90% rather than 80% power is required.

pwr.t.test(
  d = 0.50,
  sig.level = 0.05,
  power = 0.90,
  type = "two.sample",
  alternative = "two.sided"
)

The required sample size per group will increase substantially.

This illustrates an important planning tradeoff:

One-Sided R Calculation

If a one-sided test is scientifically justified:

pwr.t.test(
  d = 0.50,
  sig.level = 0.05,
  power = 0.80,
  type = "two.sample",
  alternative = "greater"
)

The required sample size will be smaller than for the corresponding two-sided test because the rejection region is placed entirely in one tail.

Changing the Clinically Meaningful Difference

Suppose the standard deviation remains 10 units but the investigators decide that a 4-unit difference is the smallest clinically meaningful effect.

Then:

Because the effect is smaller, the required sample size increases.

For a 3-unit difference:

The required sample size increases even further.

This is one of the most important relationships in sample-size planning:

Sample Size Is Not Determined by P-Value Alone

A common mistake is to ask: How many patients do we need to obtain a significant p-value?

That is not the correct planning question.

The correct question is: How many patients are required to have adequate probability of detecting a prespecified clinically meaningful difference at a prespecified type I error?

Sample-size planning therefore incorporates both statistical and clinical considerations.

Statistical Significance vs. Clinical Significance

With a sufficiently large sample, a very small treatment difference can become statistically significant.

For example:

may be rejected for a difference of only 1 unit if the study is sufficiently large and variability is sufficiently small.

That does not automatically mean a 1-unit difference is clinically important.

What Happens If the Standard Deviation Is Underestimated?

Suppose the design assumes:

but the actual standard deviation is 14.

Because:

the actual variability is substantially greater than planned.

The study will consequently have less power than expected.

This is why historical or pilot estimates of variability should be examined carefully.

What Happens If the True Effect Is Smaller?

Suppose the study is powered for:

but the actual effect is only:

Then:

instead of:

The probability of obtaining statistical significance will therefore be lower than the planned 80%.

Sample Size Under Unequal Allocation: Example

Suppose investigators want a 2:1 allocation, with twice as many patients in the treatment group as in the control group.

Let:

Then:

Using the unequal-allocation formula:

For the worked-example assumptions:

Thus approximately:

and:

for a total of approximately:

The unequal allocation therefore requires more total patients than the 1:1 allocation for the same effect, variance, alpha, and power.

Sample Size for Different Variances

Suppose the expected standard deviations are:

The planning variance becomes:

The sample-size design should then be based on the actual variance structure rather than incorrectly assuming a common standard deviation.

In such settings, validated software that matches the intended Welch-type analysis is preferable to a simplified pooled-variance formula.

Covariate Adjustment and Sample Size

Some clinical trials compare treatment groups using an analysis that adjusts for baseline covariates.

For example, an ANCOVA model may analyze a follow-up continuous endpoint while adjusting for baseline measurement.

Covariate adjustment can increase statistical efficiency when the covariates explain meaningful variation in the endpoint.

However, the sample-size calculation should be aligned with the planned primary analysis.

Change From Baseline as a Primary Endpoint

Suppose the endpoint is change from baseline:

The standard deviation used in the sample-size calculation should correspond to the variability of the analyzed change score, not necessarily the standard deviation of the post-treatment measurement alone.

This distinction can materially affect sample size.

Repeated Measures Require Additional Consideration

If the primary analysis uses multiple measurements over time, a simple two-sample t-test may not adequately represent the planned analysis.

For example, longitudinal clinical trials may use:

• Mixed-effects models
• MMRM
• Repeated-measures ANCOVA
• Generalized estimating equations

These designs involve correlation between measurements and may obtain more information from each participant than a simple comparison at one time point.

Sample-size planning should therefore reflect the actual primary analysis whenever practical.

Common Mistakes

1. Using the wrong effect size. The sample-size calculation should use the clinically meaningful difference, not an arbitrary difference selected to produce a convenient sample size.
2. Using the wrong standard deviation. The assumed SD should correspond to the endpoint and analysis being planned.
3. Confusing total N with N per group. Many formulas return the sample size per group.
4. Forgetting attrition. The analyzable sample size may need to be inflated to obtain the required number of evaluable patients.
5. Adding the dropout percentage directly. A 10% attrition rate requires division by 0.90, not simply multiplication by 1.10.
6. Confusing paired and independent designs. The variance structure is different.
7. Choosing one-sided testing solely to reduce N. The direction of the hypothesis must be scientifically justified.
8. Assuming 80% power means 80% power for every possible effect. Power is a function of the true effect size.
9. Ignoring unequal allocation. Changing the treatment-to-control ratio changes statistical efficiency.
10. Using a simple t-test calculation when the primary analysis is substantially more complex. The sample-size method should correspond as closely as practical to the planned primary analysis.

A Practical Sample Size Workflow

Worked Example Summary

What Should Be Reported in the Protocol?

A reproducible sample-size section should identify the assumptions used to derive the planned enrollment.

At minimum, report:

• Primary endpoint
• Statistical test or primary analysis model
• Clinically meaningful treatment difference
• Assumed standard deviation
• Standardized effect size, when useful
• Type I error
• One- or two-sided testing
• Target power
• Allocation ratio
• Required analyzable sample size
• Expected dropout or non-evaluable rate
• Final planned enrollment
• Software or method used for the calculation

Example Protocol Language

A concise sample-size statement could be structured around the following elements:

Why the Sample Size Calculation Should Be Reproducible

A reviewer should be able to reconstruct the sample-size calculation from the statistical documentation.

For the worked example, the essential calculation is:

followed by:

and therefore:

After allowing for 10% attrition:

This sequence makes the final enrollment transparent.

The Most Important Concept

The most important idea in two-sample t-test sample-size planning is that sample size is determined by the relationship between the clinically meaningful signal and the expected variability of the endpoint.

The central quantity is:

A larger treatment difference relative to the standard deviation means a larger signal and therefore fewer patients.

A smaller treatment difference or larger standard deviation means a smaller signal-to-noise ratio and therefore more patients.

The design must then balance that effect size against:

• Type I error
• Desired power
• Allocation ratio
• Dropout and non-evaluable rates
• Planned primary analysis
• Clinical and operational constraints

References

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Lawrence Erlbaum Associates.
Chow, S.-C., Shao, J., Wang, H. & Lokhnygina, Y. (2017). Sample Size Calculations in Clinical Research. 3rd ed. CRC Press.
Julious, S.A. (2009). Sample Sizes for Clinical Trials. CRC Press.
Machin, D., Campbell, M.J., Tan, S.B. & Tan, S.H. (2018). Sample Size Tables for Clinical Studies. 3rd ed. Wiley-Blackwell.
Senn, S. (2007). Statistical Issues in Drug Development. 2nd ed. Wiley.
van Belle, G. (2008). Statistical Rules of Thumb. 2nd ed. Wiley.