Choosing One-Sided or Two-Sided Tests for Power

Why the Test Direction Matters

A hypothesis test is designed around a research question. If the question asks whether a treatment changes an outcome in either direction, evidence of an increase or a decrease may matter. If the question asks whether the treatment improves the outcome, the direction of improvement can be specified in advance. These questions lead to different rejection regions, even when they use the same significance level and test statistic.

The choice also affects power: the probability that a test rejects the null hypothesis when a particular alternative is true. A one-sided test puts its rejection region in one tail, concentrating the chosen significance level in the direction of interest. A two-sided test divides that level between both tails. Thus, at a particular effect in the chosen direction, a one-sided test will often have greater power. That advantage comes with an important limit: it is not designed to reject for an effect in the opposite direction.

Definition: A one-sided test has an alternative hypothesis specifying a direction, such as \(H_a:\theta>\theta_0\) or \(H_a:\theta<\theta_0\). A two-sided test has an alternative hypothesis allowing either direction, \(H_a:\theta\ne\theta_0\). The rejection region is the set of test-statistic values for which the test rejects \(H_0\). The significance level \(\alpha\) is the probability of rejection when \(H_0\) is true, under the assumptions of the test.

How the Rejection Regions Differ

Consider a Z test whose statistic has a standard normal distribution under the null hypothesis. This model applies, for example, when testing a mean with known population standard deviation and independent normally distributed observations. It can also serve as an appropriate normal approximation in other settings when its conditions are met. Under a specified alternative, suppose the statistic is normally distributed with standard deviation 1 and mean \(\delta\). A positive \(\delta\) shifts the statistic toward larger values; a negative \(\delta\) shifts it toward smaller values.

For an upper-sided test, \(H_a:\theta>\theta_0\), large positive values of the statistic count against \(H_0\). For a lower-sided test, \(H_a:\theta<\theta_0\), large negative values count against \(H_0\). A two-sided test rejects for values far from zero in either direction. At \(\alpha=0.05\), the standard normal cutoffs are approximately 1.645 for an upper-sided test, -1.645 for a lower-sided test, and \(-1.960\) and 1.960 for a two-sided test.

Test and alternative	Rejection region at \(\alpha=0.05\)	Allocation of \(\alpha\)
Upper-sided: \(H_a:\theta>\theta_0\)	\(Z>1.645\)	0.05 in the upper tail
Lower-sided: \(H_a:\theta<\theta_0\)	\(Z<-1.645\)	0.05 in the lower tail
Two-sided: \(H_a:\theta\ne\theta_0\)	\(Z<-1.960\) or \(Z>1.960\)	0.025 in each tail

The reason for these cutoffs is the requirement that the probability of rejection under \(H_0\) equal \(\alpha\). For an upper-sided test, choose a cutoff \(c\) with \(P(Z>c)=0.05\) when \(Z\sim N(0,1)\). For a two-sided test, symmetry assigns half the total probability to each tail, so each tail has probability \(0.025\).

\begin{aligned} \text{Upper-sided:} \quad & P(Z>c)=\alpha,\quad c\approx1.645 \text{ when }\alpha=0.05,\\ \text{Two-sided:} \quad & P(Z<-c)+P(Z>c)=\alpha,\\ & P(Z<-c)=P(Z>c)=\alpha/2,\quad c\approx1.960 \text{ when }\alpha=0.05. \end{aligned}

This establishes the allocation directly: one-sided testing uses all of \(\alpha\) in the tail specified by \(H_a\), while two-sided testing divides \(\alpha\) equally between the tails. The more extreme two-sided cutoffs are needed because only half of the allowed null rejection probability is available in each tail.

Worked Example: Matching the Alternative to the Research Question

Suppose a hypothetical study evaluates whether a new program lowers a symptom score compared with usual care. Let \(\theta\) be the population mean score under usual care minus the population mean score under the program, so that a positive \(\theta\) represents a lower mean score with the program. State hypotheses for two possible research questions.

Question 1: Does the program change the mean score? Either an increase or a decrease could be relevant. The hypotheses are \(H_0:\theta=0\) and \(H_a:\theta\ne0\). This is a two-sided test, with a rejection region in both tails.

Question 2: Does the program lower the mean score? Under the stated definition of \(\theta\), a lower score with the program corresponds to \(\theta>0\). The hypotheses are \(H_0:\theta=0\) and \(H_a:\theta>0\). This is an upper-sided test, with rejection only for sufficiently large positive test-statistic values.

The direction follows from the question and the definition of \(\theta\), not from which way a sample estimate happens to point. If the study's goal were instead to detect possible harm as well as benefit, the two-sided question would be more appropriate.

How Direction Changes Power

For a fixed sample size and significance level, the rejection region determines the probability of detecting a particular effect. In the standard normal model above, let \(Z\sim N(\delta,1)\) under a particular alternative. Power is the probability that \(Z\) falls in the test's rejection region under that alternative. The parameter \(\delta\) describes how far the distribution is shifted from its null mean, in standard-error units.

When \(\delta>0\), an upper-sided test places its entire rejection region in the direction of the shift. A two-sided test also has an upper rejection region, but its upper cutoff is farther from zero because some of \(\alpha\) is reserved for the lower tail. At that positive alternative, this typically makes the one-sided test more likely to reject. If the true shift is negative, the relationship reverses: an upper-sided test is poorly positioned to detect it, while a two-sided test can reject in the lower tail.

Worked Example: Comparing Power for a Positive Effect

For planning, consider a hypothetical Z test at \(\alpha=0.05\). Assume the statistic is standard normal under \(H_0\), and under a particular positive alternative it follows \(Z\sim N(1.8,1)\). Compare the power of an upper-sided test with the power of a two-sided test. The assumed normal distributions and fixed standard error make this a direct comparison of rejection regions.

For the upper-sided test, reject when \(Z>1.645\). Under the alternative, subtract the alternative mean to express this probability using a standard normal variable:

\begin{aligned} \text{Power}_{\text{upper}} &=P(Z>1.645\mid Z\sim N(1.8,1))\\ &=P\left(\frac{Z-1.8}{1}> \frac{1.645-1.8}{1}\right)\\ &=P(W>-0.155)=\Phi(0.155)\approx0.5616, \end{aligned}

where \(W\sim N(0,1)\), and \(\Phi\) is the standard normal cumulative distribution function. The upper-sided test therefore has about 56.2% power at this specified alternative.

For the two-sided test, reject when \(Z<-1.960\) or \(Z>1.960\). Its power is the sum of the probabilities in those disjoint regions under the alternative:

\begin{aligned} \text{Power}_{\text{two-sided}} &=P(Z<-1.960\mid Z\sim N(1.8,1)) +P(Z>1.960\mid Z\sim N(1.8,1))\\ &=\Phi(-1.960-1.8)+1-\Phi(1.960-1.8)\\ &=\Phi(-3.760)+1-\Phi(0.160)\\ &\approx0.000085+0.4364=0.4365. \end{aligned}

At this positive effect, the one-sided test has greater power: approximately 56.2% compared with 43.7%. The comparison does not mean that one-sided tests are universally more powerful. It reflects a positive alternative in the direction chosen for the one-sided rejection region. If a negative effect were important, the upper-sided test would not provide a corresponding rejection region for it.

Choose the Direction Before Seeing the Data

The alternative hypothesis should reflect the scientific question and be set before examining the results. Choosing a one-sided test only after noticing that the observed statistic points in a convenient direction changes the intended procedure: it effectively chooses the tail after seeing the data, rather than using the tail specified by the research question. This can make the reported evidence misleading and invalidate the test's stated error rate.

A useful planning question is: Would an effect in the opposite direction still count as evidence that answers the research question? If yes, a two-sided test is generally appropriate. If the question genuinely concerns only one direction, and the opposite direction would not support the claim or decision being tested, a one-sided test may be appropriate. The study should state the direction clearly, including how the parameter is defined.

Worked Example: Interpreting a Test Statistic Without Changing the Test

Suppose a hypothetical analysis was planned as a lower-sided Z test at \(\alpha=0.05\), with \(H_a:\theta<\theta_0\). The observed statistic is \(z=-1.70\). Compare its p-values under the planned lower-sided test and under a two-sided test. Assume the standard normal null model is appropriate.

For the planned lower-sided test, the p-value is the null probability of a statistic at least as small as the observed value:

p_{\text{lower}}=P(Z\le -1.70)=0.044565\ldots\approx0.0446.

Since \(0.0446<0.05\), this result is statistically significant for the pre-specified lower-sided test.

For the two-sided test, results at least as extreme in either direction count against the null. By symmetry of the standard normal distribution:

p_{\text{two-sided}}=2P(Z\le-1.70)=2(0.044565\ldots)=0.0891309\ldots\approx0.0891.

The two-sided p-value exceeds 0.05, so the result is not statistically significant for a two-sided test at that level. The two-sided value is calculated from the unrounded tail probability; doubling the rounded value 0.0446 would give 0.0892, a slightly less accurate result. Most importantly, the analysis must use the test selected for the research question before seeing the statistic—not whichever test gives the smaller p-value afterward.

Common Mistakes and Practical Meaning

Assuming a one-sided test is always more powerful. It concentrates \(\alpha\) in one direction and can have more power at alternatives in that direction. It can have very little power against effects in the opposite direction.
Using a one-sided test because the observed result points that way. This selects the direction after seeing the data. Set the alternative from the research question before examining the results.
Thinking a one-sided test proves there is no effect in the other direction. Its rejection region does not test for an effect in the opposite direction. A result that goes that way may be important, but it does not support the specified one-sided alternative.
Putting all of \(\alpha\) in each tail of a two-sided test. That would make the total null rejection probability \(2\alpha\), not \(\alpha\). A two-sided test allocates \(\alpha/2\) to each tail.

The choice is not merely a technical route to a smaller p-value. It determines which departures from the null the analysis is designed to detect. A one-sided test may be suitable when the direction is justified in advance and the opposite direction would not answer the stated question. A two-sided test is appropriate when either direction is relevant. In both cases, the value of \(\alpha\) controls the overall probability of a Type I error under the test's assumptions.

Key takeaway: Choose a one-sided alternative only when the research question specifies a direction in advance. At a fixed \(\alpha\), a one-sided test places all rejection probability in that direction; a two-sided test splits it between both directions. The resulting power advantage is specific to alternatives in the chosen one-sided direction.

Check Your Understanding

Try these questions without looking back.

At \(\alpha=0.05\), what is the upper-tail critical value for a standard normal one-sided test, and why is the two-sided critical value farther from zero?
A study asks whether a new device changes average measurement error, with increases and decreases both considered important. Should the alternative be one-sided or two-sided? Explain.
Why can an upper-sided test have more power than a two-sided test at a specified positive alternative?
A researcher sees a negative test statistic and then changes a planned upper-sided test to a lower-sided test. What is the problem with this reasoning?
For a two-sided test with \(\alpha=0.04\), how much null rejection probability is allocated to each tail?