Integrated Power Planning for a Mean Comparison

From a Research Question to a Sample Size

A useful power plan begins with a decision, not a formula: what difference between the group means would matter enough to detect? The answer must be considered alongside the expected variability, the chance of a false positive that the study will tolerate, the desired probability of detecting the target difference, and the proposed allocation between groups. A sample-size calculation is only as credible as those choices.

This tutorial develops a complete planning calculation for two independent groups. The main formula is an approximation for a two-sided comparison with equal allocation. The same reasoning also shows how to handle unequal allocation, check sensitivity to uncertain assumptions, and account for participants who may not provide analysable data. These planning methods are aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: Let \(\mu_1\) and \(\mu_2\) be the population means for two independent groups. The target difference is \(\Delta=|\mu_1-\mu_2|\), a positive value chosen because it is scientifically or practically important. The significance level \(\alpha\) is the planned probability of rejecting a true null hypothesis, and the target power \(1-\beta\) is the planned probability of rejecting the null when the true difference is the specified target difference.

For a conventional two-sided plan, the hypotheses are \(H_0:\mu_1-\mu_2=0\) and \(H_1:\mu_1-\mu_2\ne 0\). Planning requires a value for the difference under the alternative, not merely the statement that the means might differ. It also requires a planning standard deviation, denoted \(\sigma\), and an allocation rule. In the calculations below, the two groups are independent, have a common standard deviation \(\sigma\), and are expected to follow normal distributions or have sufficiently large sample sizes for the normal approximation to be reasonable.

Equal-Allocation Planning Formula

Suppose each group will have \(n\) analysable observations. Under the common-variance model, the variance of the difference between the two sample means is \(2\sigma^2/n\), so its standard error is \(\sigma\sqrt{2/n}\). A useful normal-approximation formula for the required number per group is

n\approx \frac{2\sigma^2\left(z_{1-\alpha/2}+z_{1-\beta}\right)^2}{\Delta^2}.

Here \(z_q\) is the \(q\)-quantile of the standard normal distribution. The critical value \(z_{1-\alpha/2}\) makes the test two-sided, while \(z_{1-\beta}\) reflects the desired power. The result is a number of analysable observations per group, not a total sample size and not an enrollment target that already allows for attrition.

Why the Formula Works

Under the planning alternative, take \(\mu_1-\mu_2=\Delta>0\); the negative-difference case is symmetric. Standardizing the difference of sample means gives a statistic with approximate mean \(\Delta/(\sigma\sqrt{2/n})\) and approximate standard deviation 1. To achieve power \(1-\beta\), the mean of this standardized statistic should be about \(z_{1-\alpha/2}+z_{1-\beta}\): its mean is then \(z_{1-\beta}\) standard errors above the positive rejection threshold. Equating these quantities and solving for \(n\) gives

\frac{\Delta}{\sigma\sqrt{2/n}} \approx z_{1-\alpha/2}+z_{1-\beta} \quad\Longrightarrow\quad n\approx \frac{2\sigma^2\left(z_{1-\alpha/2}+z_{1-\beta}\right)^2}{\Delta^2}.

This argument is a planning approximation. In particular, it treats the standard deviation as the planned value and uses normal critical values. If a design will use a test with an estimated variance and modest sample sizes, the exact calculation may require a \(t\)-based power calculation. The formula is most useful as a transparent initial design calculation, not as a claim that every finite-sample test has exactly this power.

Worked Example: Equal-Allocation Design for a Target Difference

A hypothetical study will compare the mean time, in minutes, required to complete a training task under two independent training methods. A difference of 4 minutes is considered important. Planning assumes a common standard deviation of 8 minutes, a two-sided significance level of \(\alpha=0.05\), and 80% power. Find the required number of analysable participants per group.

Method. Use the equal-allocation normal approximation. The planning values are \(\Delta=4\), \(\sigma=8\), \(z_{1-\alpha/2}=z_{0.975}=1.960\), and \(z_{1-\beta}=z_{0.80}=0.842\). Thus the sum of the two normal quantiles is \(1.960+0.842=2.802\).

Substitute the values and show the arithmetic:

n\approx \frac{2(8^2)(2.802)^2}{4^2} = \frac{128(7.851204)}{16} = 62.809632.

A sample size cannot be fractional, and rounding down could miss the target. Round up to \(n=63\) analysable participants in each group, or 126 in total. Check: the calculation gives just over 62 per group, and 63 is the smallest whole-number choice that meets the calculated threshold. The common mistake avoided here is reporting 63 as the total; it is the requirement for each group.

Unequal Allocation

Equal group sizes are not always practical. Let \(n_1\) and \(n_2\) denote the analysable sample sizes in groups 1 and 2. Under independent sampling and the same common variance \(\sigma^2\), the variance of the difference in sample means is \(\sigma^2(1/n_1+1/n_2)\). Equating the planning signal-to-noise ratio to the same sum of normal quantiles gives the approximate requirement

\frac{\Delta}{\sigma\sqrt{1/n_1+1/n_2}} \approx z_{1-\alpha/2}+z_{1-\beta}.

If group 2 is planned to have \(r\) times the sample size of group 1, so that \(n_2=rn_1\), then

n_1\approx \frac{\sigma^2(1+1/r)\left(z_{1-\alpha/2}+z_{1-\beta}\right)^2}{\Delta^2}, \qquad n_2=rn_1.

This expression makes the allocation cost visible: for a fixed total sample size, a very small group can make the difference less precise. If the ratio is not a whole-number ratio after rounding, choose integer group sizes that meet the calculated requirement; do not round one group down simply to preserve a nominal ratio.

Worked Example: Planning With Twice as Many Participants in One Group

A hypothetical evaluation compares mean processing times under two systems. Assume independent groups, a common planning standard deviation of 6 seconds, a target absolute difference of 3 seconds, a two-sided \(\alpha=0.05\), and 90% power. The second system's group is planned to have twice as many analysable observations as the first. Find suitable group sizes.

Method. Set \(r=2\), use \(z_{0.975}=1.960\), and use \(z_{0.90}=1.282\). The sum is \(1.960+1.282=3.242\), with square \(3.242^2=10.510564\). The required size of group 1 is

n_1\approx \frac{6^2(1+1/2)(10.510564)}{3^2} = \frac{36(1.5)(10.510564)}{9} = 63.063384.

Round up to \(n_1=64\). Twice that size gives \(n_2=128\). These are 192 analysable observations in total. Check: the achieved allocation is exactly 1:2, and both group sizes are at least their respective planned requirements. The common mistake avoided is applying the equal-allocation formula to each group; unequal allocation changes the variance term to \(1/n_1+1/n_2\).

Sensitivity and Enrollment Allowance

The planning standard deviation and target difference are often uncertain. The formula shows how strongly they matter: the required sample size is proportional to \(\sigma^2\) and inversely proportional to \(\Delta^2\). A modest increase in the assumed standard deviation can therefore require a meaningful increase in sample size. A careful plan should calculate at least one plausible alternative set of assumptions rather than treating the primary inputs as certain.

Attrition is a separate adjustment. If a fraction \(a\) of enrolled participants is expected to be unavailable for analysis, and that fraction is assumed to be the same in each group, then the expected analysable number is the enrollment number multiplied by \(1-a\). To obtain at least \(n\) analysable participants, plan to enroll

n_{\text{enroll}}= \left\lceil \frac{n}{1-a}\right\rceil.

The ceiling symbols mean round up to the next whole participant. This calculation assumes the attrition fraction is a suitable planning expectation. If attrition differs by group, calculate each group's enrollment requirement separately.

Worked Example: Sensitivity to Variability and Allowance for Attrition

A hypothetical two-group study targets a mean difference of 3 units, with two-sided \(\alpha=0.05\), 90% power, and equal allocation. The primary planning standard deviation is 6 units, but 7 units is also plausible. Use \(z_{0.975}=1.960\) and \(z_{0.90}=1.282\). Compare the analysable sample sizes, then determine the enrollment per group if the primary design requires 80 analysable participants per group and expected attrition is 15%.

Method. The normal-quantile sum is \(1.960+1.282=3.242\), and its square is \(10.510564\). With \(\sigma=6\), the approximate requirement per group is

n\approx \frac{2(6^2)(10.510564)}{3^2} = \frac{72(10.510564)}{9} = 84.084512.

Round up to 85 analysable participants per group. Under the higher variability assumption \(\sigma=7\), the calculation becomes

n\approx \frac{2(7^2)(10.510564)}{3^2} = \frac{98(10.510564)}{9} = 114.4483636\ldots,

so the revised requirement is 115 analysable participants per group. The increase is substantial because the sample-size requirement depends on the square of the standard deviation.

For the separate attrition calculation, 80 analysable participants per group with \(a=0.15\) requires

\frac{80}{1-0.15} = \frac{80}{0.85} = 94.117647\ldots.

Thus enroll 95 per group, or 190 total. Check: \(95(0.85)=80.75\) expected analysable participants per group, whereas 94 enrolled would give \(94(0.85)=79.9\), below 80. The common mistake avoided is subtracting 15% of the target analysable sample from 80; the correct operation is to divide by the retention fraction, 0.85.

What a Complete Plan Should Report

A sample-size number is difficult to interpret without its inputs. A defensible report should state the following items explicitly:

The population means being compared and the two-sided or one-sided alternative.
The target difference \(\Delta\), in the original measurement units, and why it is meaningful.
The planning standard deviation and the population or pilot information used to choose it.
The significance level, target power, and the normal quantiles used in the calculation.
The group allocation, the required analysable sizes, and any enrollment inflation for expected attrition.
At least one sensitivity calculation for a plausible change in the target difference or variability.

A useful diagnostic is to inspect the direction of each change. A smaller target difference requires a larger sample; a larger standard deviation requires a larger sample; more stringent significance or higher power also increases the requirement. If a calculation behaves in the opposite direction, recheck the inputs and algebra. The normal approximation should also be treated cautiously when sample sizes are small, the distributions are strongly nonnormal, the common-variance assumption is not credible, or observations are not independent. Those conditions call for a design model and power calculation that match the intended analysis rather than blind use of this formula.

Key takeaway: A complete mean-comparison plan links a practically meaningful difference to variability, \(\alpha\), power, allocation, and attrition. State which numbers are analysable sample sizes and which are enrollment targets, and check sensitivity before treating the result as fixed.

Check Your Understanding

For each problem, use the normal-approximation planning formula where applicable, round sample sizes up, and distinguish analysable counts from enrollment counts.

An equal-allocation comparison targets a difference of 2.5 units. Assume \(\sigma=5\), two-sided \(\alpha=0.05\), and 80% power. Use \(z_{0.975}=1.960\) and \(z_{0.80}=0.842\). What is the required sample size per group? (Answer: 63 per group.)
An equal-allocation design targets a difference of 2 units with \(\sigma=4\), two-sided \(\alpha=0.01\), and 90% power. Use \(z_{0.995}=2.576\) and \(z_{0.90}=1.282\). What is the required sample size per group? (Answer: 120 per group.)
A study requires 120 analysable participants in each group and expects 10% attrition in both groups. How many should be enrolled per group and in total? (Answer: 134 per group, 268 total.)
An equal-allocation plan targets a difference of 2 units with \(\sigma=6\), two-sided \(\alpha=0.05\), and 90% power. Use \(z_{0.975}=1.960\) and \(z_{0.90}=1.282\). What is the required sample size per group? (Answer: 190 per group.)
For an unequal-allocation design, group 2 is planned to be twice the size of group 1. Assume \(\sigma=6\), target difference 3, two-sided \(\alpha=0.05\), and 90% power, using a quantile sum of 3.242. Calculate the rounded group sizes. (Answer: 64 in group 1 and 128 in group 2.)

What You'll Learn