Planning Sample Size for a Mean Test

From a Meaningful Difference to a Sample Size

Before collecting data, a researcher needs to decide how many observations to obtain. For a one-sample mean test, that decision depends on the difference the study should be able to detect, the variability of individual observations, the significance level, and the desired power. A larger sample makes the sample mean more precise, but it also requires more time, participants, or resources.

The difference used in planning should be scientifically meaningful, not merely a value chosen because it produces a convenient sample size. For example, a clinician might decide that a change of 5 points on a symptom scale would matter to patients. The calculation then asks how many independent measurements are needed for a test to have a specified chance of detecting a true difference of that size.

The standard deviation estimate should come from relevant prior information, such as earlier measurements taken in a similar population using the same scale. It describes the spread of individual observations, not the spread of the sample mean. The calculation uses it together with the planned difference to determine how many observations are needed.

The Normal-Based Planning Formula

Suppose observations are independent and come from a population with standard deviation \(\sigma\). If \(\sigma\) is known and the population is normal, the sample mean has distribution \(\bar{x}\sim N(\mu,\sigma^2/n)\). With a sufficiently large sample, this normal model for the sample mean can also be a useful approximation under less restrictive population shapes. For planning, \(\sigma\) is usually an estimate rather than a known value.

Write \(z_q\) for the \(q\)-quantile of the standard normal distribution, so that \(P(Z\le z_q)=q\) when \(Z\sim N(0,1)\). For a two-sided test of \(H_0:\mu=\mu_0\) against \(H_a:\mu\ne\mu_0\), the approximate required sample size is:

For a one-sided test, the corresponding approximation is:

In either case, round the result up to the next whole number. A sample cannot contain a fraction of an observation, and rounding down could leave the planned power below its target. The two-sided formula uses a more extreme critical value because the rejection region is split between both tails.

Why the Formula Works

The formula follows by expressing the test statistic in standard deviation units under the alternative mean. For a two-sided z test with \(\sigma\) known, reject \(H_0\) when the absolute value of the test statistic exceeds \(z_{1-\alpha/2}\). If \(\mu_1>\mu_0\), the test statistic has a standard normal distribution shifted to the right by \(\lambda=\delta\sqrt{n}/\sigma\). Its power is therefore:

For a positive shift, the second probability is often small at the design effect of interest. The first term reaches the target power \(1-\beta\) when \(z_{1-\alpha/2}-\lambda=-z_{1-\beta}\). Solving this equation gives \(\lambda=z_{1-\alpha/2}+z_{1-\beta}\). Substituting \(\lambda=\delta\sqrt{n}/\sigma\) and squaring both sides gives the two-sided planning formula. The one-sided formula follows by replacing the two-sided critical value with \(z_{1-\alpha}\).

This derivation also shows what drives sample size: the effect relative to the standard deviation, \(\delta/\sigma\). When the meaningful difference is small compared with the spread of individual measurements, a larger sample is needed. The formula is an approximation, especially for small samples and for tests that estimate \(\sigma\) from the same data.

Choosing the Test Direction and Applying the Plan

The one-sided formula is appropriate only when the scientific question and test direction are fixed before examining the results. A one-sided test puts the full significance level in one tail, so it can require fewer observations for a specified effect in that direction. It does not provide the same test for a difference in the opposite direction. Choosing a one-sided test after seeing which way the sample mean moved would invalidate the stated error rate.

Planning When the Standard Deviation Is Estimated

In many mean tests, the population standard deviation is unknown and the final analysis uses a one-sample t test. Its statistic is \(T=(\bar{x}-\mu_0)/(s/\sqrt{n})\), where \(s\) is the sample standard deviation. A standard normal calculation is then a planning approximation, not an exact t-test sample-size calculation. It is often a useful first calculation when the sample is not very small and a reasonable estimate of \(\sigma\) is available.

For an exact power calculation under a normal population, the t statistic under a specified alternative follows a noncentral t distribution with \(n-1\) degrees of freedom and noncentrality parameter \(\lambda=(\mu_1-\mu_0)\sqrt{n}/\sigma\). Because both the degrees of freedom and the noncentrality parameter depend on \(n\), the required sample size is typically found by checking candidate sample sizes numerically. For a two-sided test, power is the probability that this statistic falls below the negative t critical value or above the positive t critical value. A numerical t-based calculation can refine the normal-based estimate, particularly when the planned sample is small.

Common Planning Mistakes

• Using an arbitrary difference. A sample-size calculation cannot decide which effect matters scientifically. Specify the smallest meaningful difference first, in the measurement's original units.
• Using a standard error in place of a standard deviation. The formula takes the spread of individual observations. The standard error depends on sample size and is \(\sigma/\sqrt{n}\); it is not the input estimate for \(\sigma\).
• Rounding down. If the calculation gives 60.54, a plan for 60 observations does not meet the calculated threshold. Round up to 61.
• Switching to a one-sided test after seeing the data. The direction must be justified and chosen in advance. Otherwise, the stated significance level no longer describes the test as planned.
• Treating an approximation as an exact guarantee. The result depends on the assumed variability, effect size, model, and test. A t-based refinement may be needed, and real data may differ from the planning assumptions.