Calculating Power for a Two-Sided Z Test

Why Two-Sided Power Uses Two Tails

A two-sided test can reject the null hypothesis when the observed sample mean is either sufficiently below or sufficiently above the null mean. To calculate its power, therefore, we must find the probability of landing in either rejection region when a particular alternative mean is true. That probability includes both tails, even when the alternative of interest lies in only one direction.

Power is not a single number determined by the test alone. It depends on the specified alternative mean, as well as on the sample size, the population standard deviation, and the significance level. The calculation below fixes those quantities and asks how often the test would reject under that particular alternative.

Definition: The power of a hypothesis test at a specified alternative is the probability that the test rejects \(H_0\) when that alternative is true. For a two-sided one-sample z test of \(H_0:\mu=\mu_0\) against \(H_a:\mu\ne\mu_0\), power at a particular true mean \(\mu_1\) is the probability of rejecting \(H_0\) when \(\mu=\mu_1\).

The Two Rejection Cutoffs

Suppose the observations form a random, independent sample of size \(n\) from a population with mean \(\mu\) and known standard deviation \(\sigma\). For an exact z-test calculation, assume the population is normally distributed; for a sufficiently large sample, the normal model for \(\bar{x}\) may instead be justified approximately. The standard error is \(SE=\sigma/\sqrt{n}\). Under the null hypothesis, the test statistic is \(Z=(\bar{x}-\mu_0)/SE\), which has a standard normal distribution.

Let \(z_{1-\alpha/2}\) be the standard normal value with cumulative probability \(1-\alpha/2\) to its left. The two-sided test rejects \(H_0\) when \(Z<-z_{1-\alpha/2}\) or \(Z>z_{1-\alpha/2}\). In terms of the sample mean, the rejection regions are below the lower cutoff or above the upper cutoff:

\bar{x}<\mu_0-z_{1-\alpha/2}SE \qquad\text{or}\qquad \bar{x}>\mu_0+z_{1-\alpha/2}SE

The cutoffs are set using the null model, not the alternative. Under \(H_0\), each tail has probability \(\alpha/2\), so the total probability of rejection is \(\alpha\). To calculate power, we keep these same cutoffs and find the probabilities of crossing them under the alternative distribution.

Worked Example: Power When the Alternative Mean Is Above the Null

Suppose a hypothetical random sample of 36 independent measurements comes from a normal population with known \(\sigma=15\) units. Test \(H_0:\mu=100\) units against \(H_a:\mu\ne100\) units at \(\alpha=0.05\). Calculate power if the true mean is \(\mu_1=104\) units.

The stipulated normal population and independent random sampling meet the model conditions for this calculation, and the population standard deviation is known. The standard error is \(SE=15/\sqrt{36}=2.5\) units. For a two-sided test with \(\alpha=0.05\), \(z_{1-\alpha/2}=z_{0.975}\approx1.96\). The cutoffs for \(\bar{x}\) are:

100\pm1.96(2.5)=100\pm4.9, \qquad 95.1\text{ units and }104.9\text{ units}.

When the true mean is 104 units, the sampling distribution of \(\bar{x}\) is normal with mean 104 units and standard deviation 2.5 units. Standardizing each cutoff under this alternative gives:

P(\bar{x}<95.1)+P(\bar{x}>104.9) = \Phi\!\left(\frac{95.1-104}{2.5}\right) + 1-\Phi\!\left(\frac{104.9-104}{2.5}\right)

The two standardized values are \(-3.56\) and \(0.36\). Using standard normal probabilities, \(\Phi(-3.56)\approx0.0002\) and \(1-\Phi(0.36)\approx0.3594\). Thus the power is approximately \(0.0002+0.3594=0.3596\), or 36.0%. If the true mean were 104 units and this test were repeated under the stated model, it would reject \(H_0\) about 36.0% of the time.

The lower-tail contribution is small here, but it is part of the two-sided test’s power and has been included.

Calculating Both Probabilities Under an Alternative

Write \(\Phi(a)\) for the probability that a standard normal random variable is less than \(a\). Under a particular true mean \(\mu_1\), the sample mean has distribution \(N(\mu_1,\sigma^2/n)\) when the population is normal. The lower-tail probability is the chance that \(\bar{x}\) falls below the lower cutoff; the upper-tail probability is the chance that it exceeds the upper cutoff. These regions do not overlap, so their probabilities add.

Equivalently, standardize the sample mean relative to the null mean and its standard error. Under \(\mu_1\), this standardized statistic has variance 1 but is centered at \((\mu_1-\mu_0)/SE\), rather than at zero. Substituting the two rejection cutoffs into that distribution gives the power formula:

Power formula: For a two-sided one-sample z test with known \(\sigma\), significance level \(\alpha\), and specified true mean \(\mu_1\), let \(c=z_{1-\alpha/2}\) and \(SE=\sigma/\sqrt{n}\). Then

\text{Power}(\mu_1) = \Phi\!\left(\frac{\mu_0-cSE-\mu_1}{SE}\right) + 1-\Phi\!\left(\frac{\mu_0+cSE-\mu_1}{SE}\right)

Both terms are required: the first is the probability in the lower rejection region, and the second is the probability in the upper rejection region.

A useful equivalent expression sets \(\delta=(\mu_1-\mu_0)/SE\), the alternative mean’s distance from the null in standard-error units. If \(T=(\bar{x}-\mu_0)/SE\), then under \(\mu_1\), \(T\) has a normal distribution with mean \(\delta\) and variance 1. Therefore:

\text{Power}(\mu_1) =P(T<-c)+P(T>c) =\Phi(-c-\delta)+1-\Phi(c-\delta).

Justifying the Formula

The formula follows by expressing each part of the rejection event as a probability under the alternative. Because \(\bar{x}\) has mean \(\mu_1\) and standard deviation \(SE\), standardizing a cutoff \(a\) gives \((a-\mu_1)/SE\). Applying this to the lower and upper cutoffs yields:

Write the rejection event.
The test rejects if \(\bar{x}<\mu_0-cSE\) or \(\bar{x}>\mu_0+cSE\). These two events are disjoint.

Use the alternative distribution.
When the true mean is \(\mu_1\), \(\bar{x}\sim N(\mu_1,SE^2)\). Thus the probability of falling below a cutoff \(a\) is \(\Phi((a-\mu_1)/SE)\).

Add the tail probabilities.
The lower event contributes \(\Phi((\mu_0-cSE-\mu_1)/SE)\). The upper event contributes \(1-\Phi((\mu_0+cSE-\mu_1)/SE)\). Their sum is the probability of rejection under \(\mu_1\), which is power.

The use of \(1-\Phi(\cdot)\) for the upper tail matters: \(\Phi(a)\) gives the probability to the left of \(a\), not to its right. Also, the alternative distribution is centered at \(\mu_1\), while the rejection cutoffs remain the ones chosen under \(H_0\).

Worked Example: A Nonzero Null Mean

In a hypothetical setting, an independent random sample of 64 measurements is taken from a normal population with known \(\sigma=8\) units. Consider \(H_0:\mu=50\) units versus \(H_a:\mu\ne50\) units at \(\alpha=0.01\). Find the power when the true population mean is 53 units.

The stated normal model, independent sampling, and known \(\sigma\) support a one-sample z calculation. The standard error is \(8/\sqrt{64}=1\) unit. The critical value for \(\alpha=0.01\) is \(z_{0.995}\approx2.576\). The rejection cutoffs are \(50\pm2.576(1)\), or 47.424 and 52.576 units.

Under the alternative mean of 53 units, standardize both cutoffs using a normal distribution centered at 53 with standard deviation 1:

\text{Power} = \Phi\!\left(\frac{47.424-53}{1}\right) + 1-\Phi\!\left(\frac{52.576-53}{1}\right) = \Phi(-5.576)+1-\Phi(-0.424).

The lower-tail probability is negligible, approximately \(0.00000001\). By symmetry of the standard normal distribution, \(1-\Phi(-0.424)=\Phi(0.424)\approx0.6642\). The total power is therefore approximately \(0.6642\), or 66.4%. The null mean being nonzero changes the cutoffs’ location, but not the calculation: both cutoffs are measured from the alternative mean using the standard error.

When the Alternative Is Below the Null

Power still includes both rejection tails when the specified alternative is below \(\mu_0\). In that case the lower tail will usually contribute more, but the upper tail is not removed from the test. This can be seen clearly when the alternative is close enough to the lower cutoff that some of its sampling distribution remains on either side.

Worked Example: Adding the Smaller Tail

Suppose a hypothetical independent sample of 36 observations comes from a normal population with known \(\sigma=6\) units. Test \(H_0:\mu=30\) units against \(H_a:\mu\ne30\) units at \(\alpha=0.10\). Calculate power if the true mean is 28.5 units.

The normal population, independent observations, and known standard deviation satisfy the z-test assumptions. Here \(SE=6/\sqrt{36}=1\) unit, and \(z_{0.95}\approx1.645\). The rejection cutoffs are \(30-1.645(1)=28.355\) and \(30+1.645(1)=31.645\) units.

Under the alternative, \(\bar{x}\) has mean 28.5 units and standard deviation 1 unit. The lower and upper probabilities are:

\text{Power} = \Phi\!\left(\frac{28.355-28.5}{1}\right) + 1-\Phi\!\left(\frac{31.645-28.5}{1}\right) = \Phi(-0.145)+1-\Phi(3.145).

From the standard normal distribution, \(\Phi(-0.145)\approx0.4424\), and \(1-\Phi(3.145)\approx0.00083\). Adding the two contributions gives approximately \(0.4424+0.00083=0.44323\), which rounds to 0.4432 (about 44.3%). Most of the power comes from the lower rejection region, but the small upper-tail probability must still be included.

What Changes Power?

For fixed \(\alpha\), \(n\), and \(\sigma\), power is determined by how far the specified alternative mean lies from \(\mu_0\) in standard-error units. In the formula, that distance is \(\delta=(\mu_1-\mu_0)/SE\). If \(\mu_1=\mu_0\), then \(\delta=0\) and the probability of rejection is \(\alpha\). As the alternative mean moves farther from the null in either direction, more of its sampling distribution lies in the two rejection regions.

Because the rejection regions are symmetric around \(\mu_0\) and the standard normal distribution is symmetric, alternatives equally far above and below \(\mu_0\) have the same power in this model. For example, changing \(\mu_1-\mu_0\) from a positive value to its negative leaves power unchanged. The individual lower- and upper-tail contributions swap roles.

Common Mistakes

Counting only the tail in the direction of the alternative. A two-sided test rejects in both tails, so power is the sum of both probabilities under the alternative.
Using the null mean to center the alternative probabilities. The null mean determines the critical cutoffs; the specified alternative mean centers the distribution used to calculate power.
Using the wrong side of a normal probability. The upper-tail probability beyond a standardized cutoff \(a\) is \(1-\Phi(a)\), not \(\Phi(a)\).
Interpreting power as the chance the null hypothesis is false. Power is a rejection probability calculated under a specified alternative; it is not a probability assigned to either hypothesis.
Reporting power without specifying the alternative. A statement such as “the test has 80% power” is incomplete unless the alternative mean and the test assumptions are stated.

Key takeaway: For a two-sided one-sample z test, first set both critical cutoffs under \(H_0\). Then, under the specified alternative mean, calculate the probability below the lower cutoff and the probability above the upper cutoff, and add them. The result is power at that alternative.

Check Your Understanding

Try these questions before reviewing the formulas.

For a two-sided z test with \(\alpha=0.05\), which standard normal critical value determines the two rejection cutoffs?
Why are the rejection cutoffs calculated from \(\mu_0\), while the power probabilities are calculated using a distribution centered at \(\mu_1\)?
If the specified alternative mean is below the null mean, should power include the upper-tail probability? Explain.
For a test with known \(\sigma\), what happens to the standard error when the sample size increases, with \(\sigma\) held fixed?
In a two-sided z test, what does a power of 0.70 at \(\mu_1\) mean in context?