Directional Alternatives and One-Sided Tests

Why the Direction Must Be Chosen Carefully

When comparing two population means, the research question may predict a direction. A new process might be expected to increase average output, or a treatment might be expected to reduce average recovery time. A one-sided test addresses such a directional question. Its direction determines which sample outcomes count as evidence against the null hypothesis, so it must come from the question posed before looking at the observed difference—not from the direction the data happen to point.

Throughout this tutorial, group 1 and group 2 are named in that order, and the parameter of interest is \(\mu_1-\mu_2\). Thus, a positive difference means that the population mean for group 1 is greater than the population mean for group 2. Reversing the group order reverses the sign and changes which alternative is appropriate.

Definition: A one-sided alternative specifies a direction for the difference between two population means. For \(\delta=\mu_1-\mu_2\), the alternatives are \(H_a:\delta>0\) (group 1 has the larger mean) or \(H_a:\delta<0\) (group 1 has the smaller mean). The corresponding null includes equality and all differences in the opposite direction: \(H_0:\delta\leq 0\) for \(H_a:\delta>0\), or \(H_0:\delta\geq 0\) for \(H_a:\delta<0\). The test is calibrated at the boundary value \(\delta=0\).

The Welch Test and Its Directional P-Value

A one-sided alternative changes the tail of the test, not the basic Welch standard error or test statistic. The procedure compares two independent groups with quantitative outcomes and does not require the population variances to be equal. Its assumptions are that observations are independent within each group and between groups, and that the data come from random samples or an appropriate randomized design. For small samples, each population should be approximately normal, without severe skewness or influential outliers; larger samples can make the procedure more robust, but serious dependence or extreme outliers remain concerns. These are the conditions for reliable two-sample t inference.

Let \(\bar{x}_1,\bar{x}_2\) be the sample means, \(s_1,s_2\) the sample standard deviations, and \(n_1,n_2\) the sample sizes. Under the boundary null hypothesis of equal means, the Welch statistic and its approximate degrees of freedom are:

SE=\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}, \qquad t=\frac{(\bar{x}_1-\bar{x}_2)-0}{SE}

df \approx \frac{\left(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}\right)^2} {\frac{(s_1^2/n_1)^2}{n_1-1}+\frac{(s_2^2/n_2)^2}{n_2-1}}

The Welch reference distribution is a t distribution with this approximate \(df\). For \(H_a:\mu_1-\mu_2>0\), the p-value is the upper-tail probability \(P(T_{df}\geq t)\). For \(H_a:\mu_1-\mu_2<0\), it is the lower-tail probability \(P(T_{df}\leq t)\). The same observed statistic can therefore give very different p-values for opposite one-sided alternatives.

A p-value is calculated assuming the null hypothesis is true at its boundary, here \(\mu_1-\mu_2=0\). It is the probability, under that assumption and the model, of obtaining a statistic at least as far in the direction specified by \(H_a\) as the observed statistic. A small p-value indicates that the observed result is unusual under the null in that particular direction; it is not the probability that the null is true.

Worked Example: Testing Whether One Process Has Higher Output

In a hypothetical evaluation, 12 independently selected production runs use process 1 and 10 independent runs use process 2. Output is measured in units per hour. The sample means are 14.2 and 9.2 units per hour, and the sample standard deviations are 4.0 and 6.0 units per hour, respectively. Before collecting data, the investigators specified that process 1 should produce higher output. Test at \(\alpha=0.05\).

Hypotheses and conditions. In the stated group order, \(H_0:\mu_1-\mu_2\leq0\) and \(H_a:\mu_1-\mu_2>0\). The runs are described as independent; assume they were selected or assigned in a way that supports inference to the target process conditions. With only 10 and 12 runs, the output distributions should be checked for severe skewness or outliers, and should be reasonably well behaved. No equal-variance assumption is needed for Welch's procedure.

Standard error and statistic.

SE=\sqrt{\frac{4.0^2}{12}+\frac{6.0^2}{10}} =\sqrt{1.3333+3.6000} =\sqrt{4.9333} \approx 2.221\text{ units per hour}

t=\frac{(14.2-9.2)-0}{2.221} =\frac{5.0}{2.221} \approx 2.251

Degrees of freedom and p-value.

df \approx \frac{(1.3333+3.6000)^2} {1.3333^2/11+3.6000^2/9} \approx 15.20

For the prespecified greater-than alternative, the p-value is the upper-tail probability \(P(T_{15.20}\geq2.251)\), approximately 0.020. Since 0.020 is less than 0.05, reject \(H_0\) at the 5% significance level. The data provide evidence that process 1 has higher mean output than process 2 under the conditions studied. This conclusion does not establish that the difference is practically important or that process 1 will be higher in every setting.

Choose the Tail From the Question

Writing the group order and alternative in words is a useful safeguard. If group 1 is expected to have a smaller mean, use \(H_a:\mu_1-\mu_2<0\) and the lower tail. If it is expected to have a larger mean, use \(H_a:\mu_1-\mu_2>0\) and the upper tail. The sign of the calculated \(t\) statistic describes the sample difference; it does not decide which alternative is scientifically appropriate.

The direction should be justified by the research question, subject-matter reasoning, or a plan made before examining the outcomes. The statement “we will test whether group 1 is different from group 2” is not directional; it calls for a two-sided alternative. A directional test is suitable only when evidence in the opposite direction would not count as support for the specific claim being tested.

Worked Example: A Prespecified Reduction in Recovery Time

In a hypothetical study, group 1 receives a new care protocol and group 2 receives standard care. Recovery time is measured in days. The protocol was expected, before the study, to reduce mean recovery time. Independent samples produce \(\bar{x}_1=42.0\), \(s_1=8.0\), \(n_1=9\) and \(\bar{x}_2=46.0\), \(s_2=5.0\), \(n_2=14\). Assume the design supports independent observations and, because the samples are small, that the recovery-time distributions have no severe skew or influential outliers.

The prespecified alternative is \(H_a:\mu_1-\mu_2<0\), with null \(H_0:\mu_1-\mu_2\geq0\). The sample difference is \(42.0-46.0=-4.0\) days. The Welch standard error is:

SE=\sqrt{\frac{8.0^2}{9}+\frac{5.0^2}{14}} =\sqrt{7.1111+1.7857} =\sqrt{8.8968} \approx2.983\text{ days}

The test statistic and approximate degrees of freedom are:

t=\frac{(42.0-46.0)-0}{2.983} \approx -1.341

df\approx \frac{(7.1111+1.7857)^2} {7.1111^2/8+1.7857^2/13} \approx12.06

Because the alternative is less-than, use the lower-tail probability \(P(T_{12.06}\leq-1.341)\), approximately 0.102. At \(\alpha=0.05\), this is not statistically significant: the data do not provide sufficient evidence that the new protocol reduces mean recovery time. The sample mean is 4.0 days lower for group 1, but a difference in sample means alone does not establish the directional population claim.

Why Selecting a Direction After Seeing the Data Is a Problem

Choosing the tail after seeing the sign of the sample difference gives the data two chances to look convincing: a sufficiently large positive statistic can be treated as evidence in the upper tail, while a sufficiently large negative statistic can be treated as evidence in the lower tail. This is not the same test as one whose direction was fixed in advance.

The increase in false-positive risk can be seen directly. Under the boundary null, the Welch reference t distribution is symmetric around zero. Suppose a researcher selects the direction that matches the observed sign and then rejects at one-sided level \(\alpha\). The rejection rule is effectively to reject for a statistic far enough into either tail:

P(\text{reject }H_0\mid H_0\text{ at boundary}) = P(T_{df}\geq t_{1-\alpha,df}) + P(T_{df}\leq -t_{1-\alpha,df}) =\alpha+\alpha=2\alpha

Here \(t_{1-\alpha,df}\) is the upper-tail critical value for a one-sided test at level \(\alpha\). The equality follows from the symmetry of the reference distribution: each tail beyond the corresponding cutoff has probability \(\alpha\). Thus, selecting whichever direction looks favorable after seeing the data can make the false-positive probability about twice the intended one-sided level.

Worked Example: What Post-Data Direction Selection Does

Return to the hypothetical production data above, which gave \(t=2.251\) and \(df\approx15.20\). If “process 1 has higher output” was specified in advance, the correct one-sided p-value is approximately 0.020. If the direction was chosen only after noticing that the sample mean for process 1 was larger, that one-sided p-value does not reflect the selection.

A two-sided test instead asks whether the means differ in either direction. For this statistic, its p-value is approximately twice the one-tail probability, or \(2(0.020)\approx0.040\). More generally, at a one-sided cutoff of \(\alpha=0.05\), selecting the tail after seeing the sign rejects when the statistic falls into either 5% tail. Under the boundary null, the total rejection probability is approximately \(0.05+0.05=0.10\), not 0.05.

The appropriate conclusion depends on the plan: use the one-sided test if its direction was genuinely prespecified; use a two-sided test when either direction would answer the question or when no direction was set in advance. Do not describe a direction chosen after inspecting the results as prespecified.

Common Mistakes and a Practical Checklist

Choosing a tail from the observed sign. A positive sample difference does not by itself justify a greater-than alternative. Set the direction from the question before looking at the outcomes.
Using the wrong group order. If the parameter is \(\mu_1-\mu_2\), then a negative value means group 1 has the smaller mean. State which group is group 1 and keep that order throughout.
Using the wrong tail. For \(H_a:\mu_1-\mu_2>0\), calculate the upper-tail probability; for \(H_a:\mu_1-\mu_2<0\), calculate the lower-tail probability. A statistic in the direction opposite to the alternative will generally have a large p-value.
Interpreting the p-value as the chance the null is true. It is a probability of data at least as extreme in the specified direction under the null model, not a probability assigned to a hypothesis.
Assuming equal population variances. Welch's standard error and degrees of freedom use the two sample variances separately. They do not require a pooled variance.

Before calculating a one-sided Welch p-value, write the group order, the difference \(\mu_1-\mu_2\), the alternative in words and symbols, and the reason that direction was chosen in advance. Then verify that the independence and distribution conditions are reasonable, calculate \(t\) and \(df\), and use the tail specified by \(H_a\). Report the conclusion in terms of the two population means and the measurement units.

Check Your Understanding

Answer each question using the group order and direction carefully.

For \(\mu_1-\mu_2\), write the null and alternative hypotheses if the prespecified claim is that group 1 has a larger population mean.
A study's alternative is \(H_a:\mu_1-\mu_2<0\), but the observed Welch statistic is positive. Which tail probability is required, and would the result ordinarily be strong evidence for the alternative?
In a one-sided Welch test with \(t=1.8\), \(df=14\), and \(H_a:\mu_1-\mu_2>0\), describe in words what the p-value measures.
Why can choosing the direction after seeing the sample means raise the false-positive rate above the stated one-sided significance level?
What additional issue should be checked when the two sample sizes are small, even if the observations are independent?