Choosing Which Outcomes Count as Extreme

Introduction

A test statistic summarizes how the observed data compare with a null hypothesis. To determine which results count as “at least as extreme” as the observed result, however, the statistic alone is not enough. The alternative hypothesis tells us what kind of departure from the null is relevant: a larger value, a smaller value, or a departure in either direction.

This choice determines the event whose probability is used for the p-value. A right-tailed alternative calls for outcomes at least as large as the observed statistic; a left-tailed alternative calls for outcomes at least as small. With a two-sided alternative, departures in both directions count. The direction must come from the research question and hypotheses, not be chosen after seeing which tail makes the result look stronger.

Definition: Given a test statistic \(T\), its observed value \(t_{\text{obs}}\), a null reference distribution, and an alternative hypothesis, the outcomes “at least as extreme” are the values of \(T\) that are at least as far in the direction or directions specified by the alternative as \(t_{\text{obs}}\). The p-value is the probability, under the null model, of the specified extreme-outcome event.

Let the Alternative Determine the Tail

For a test about a population mean \(\mu\), a common statistic is a standardized difference between a sample mean \(\bar{x}\) and the null value \(\mu_0\). For example, when the population standard deviation \(\sigma\) is known, the statistic is \(Z=(\bar{x}-\mu_0)/(\sigma/\sqrt{n})\). Because the denominator is positive, larger values of \(Z\) correspond to larger sample means, and smaller values correspond to smaller sample means.

The same logic applies to a t statistic, or to another statistic whose direction is defined so that larger and smaller values represent the corresponding directions of evidence. The alternative determines which side of the observed statistic is relevant.

Alternative hypothesis	Outcomes at least as extreme as \(t_{\text{obs}}\)	Relevant direction
Parameter is greater than its null value	\(T \ge t_{\text{obs}}\)	Right tail
Parameter is less than its null value	\(T \le t_{\text{obs}}\)	Left tail
Parameter differs from its null value	Values at least as far from the null in either direction	Both tails

“At least as extreme” includes the observed statistic itself and continues farther in the relevant direction. In a one-sided test, that means moving to the right for a greater-than alternative or to the left for a less-than alternative. It does not mean choosing whichever side has the larger probability or whichever side happens to contain the observation.

Right-Tailed Alternatives

Suppose the alternative says the population mean is greater than the value specified by the null hypothesis. Evidence in that direction corresponds to a large positive standardized statistic. If the observed statistic is \(z_{\text{obs}}\), the relevant event is \(Z \ge z_{\text{obs}}\), and the p-value is \(P(Z \ge z_{\text{obs}})\) under the null reference distribution.

Worked Example: A Right-Tailed Test for a Mean

A hypothetical process has a null mean of 100 units. A random sample of 25 measurements has mean 104 units. Suppose the population standard deviation is known to be 10 units, the measurements are independent, and the population is normally distributed. The question is whether the population mean exceeds 100 units, so \(H_0:\mu=100\) and \(H_a:\mu>100\).

Check the model. The sample is described as random, its measurements as independent, and the population as normally distributed. With known \(\sigma=10\), these conditions justify the null model \(Z\sim N(0,1)\) for the standardized sample mean when \(H_0\) is true.

Calculate the observed statistic.

z_{\text{obs}} = \frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}} = \frac{104-100}{10/\sqrt{25}} = \frac{4}{2} = 2

Select the extreme outcomes. The alternative is \(\mu>100\), so only values at least as large as the observed statistic count as extreme: \(Z\ge 2\). Under the null model,

P(Z\ge 2)=0.0228

The p-value is approximately 0.0228. It is the null-model probability of a standardized sample mean of 2 or greater for a test whose alternative is that the population mean exceeds 100 units. The left tail is not added: values far below 0 would point away from this particular alternative.

Left-Tailed Alternatives

If the alternative says the population mean is less than the null value, evidence in its direction corresponds to a small statistic. For an observed value \(t_{\text{obs}}\), the relevant event is \(T\le t_{\text{obs}}\). In particular, a negative observed statistic often points in the direction of a less-than alternative when the statistic is formed as “estimate minus null value,” divided by a positive standard error.

Worked Example: A Left-Tailed Test Using a t Statistic

In a hypothetical setting, a random sample of 10 independent measurements has mean 18.2 units and sample standard deviation 3 units. Assume the population is normally distributed. Test whether its mean is below 20 units: \(H_0:\mu=20\) and \(H_a:\mu<20\).

Check the model. The measurements are assumed to be a random independent sample, and the population is assumed normal. Since the population standard deviation is not given, use the sample standard deviation in a one-sample t statistic with \(df=n-1=9\). Under \(H_0\), the reference statistic follows a t distribution with 9 degrees of freedom.

Calculate the statistic and identify the event.

t_{\text{obs}} = \frac{\bar{x}-\mu_0}{s/\sqrt{n}} = \frac{18.2-20}{3/\sqrt{10}} = \frac{-1.8}{0.9487} \approx -1.897

Because the alternative is \(\mu<20\), outcomes at least as extreme are \(T\le -1.897\), not values in the right tail. The p-value is therefore

P(T_9\le -1.897)\approx 0.045

Under the stated t model and null hypothesis, the probability of a t statistic at or below \(-1.897\) is approximately 0.045. The tail follows the alternative’s direction; the fact that the observed statistic is negative is consistent with, but does not itself determine, the choice of a left-tailed test.

Two-Sided Alternatives

An alternative such as \(\mu\ne\mu_0\) treats departures above and below the null value as relevant. For a signed standardized statistic centered at zero under the null, an observation far to the left and one equally far to the right are both departures in the alternative’s direction. Thus the extreme-outcome event has a lower-tail part and an upper-tail part.

The point here is to identify the outcomes that count, not to calculate the probability of their combined event. Constructing two-sided p-values requires an additional calculation and careful attention to the reference distribution. The next example therefore identifies the event but does not evaluate its probability.

Worked Example: Identifying Both Tails

A hypothetical sample of 36 independent measurements has mean 47.1 units. Assume a normal population with known standard deviation 6 units. The question is whether the population mean differs from 50 units: \(H_0:\mu=50\) and \(H_a:\mu\ne50\).

Check the model. The sample is assumed random and independent, the population is normal, and the population standard deviation is known. Under \(H_0\), the standardized statistic has a standard normal reference distribution.

Find the observed statistic.

z_{\text{obs}} = \frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}} = \frac{47.1-50}{6/\sqrt{36}} = \frac{-2.9}{1} = -2.9

Since the alternative is “differs from 50,” extreme values include those at least as far below zero as \(-2.9\) and those at least as far above zero as \(2.9\). The event is

Z\le -2.9 \quad \text{or} \quad Z\ge 2.9

The observed value lies in the lower tail, but the alternative also makes equally distant upper-tail outcomes relevant. Considering only the lower tail would answer a different, one-sided question.

Why the Tail Choice Matters

The p-value is a probability of outcomes defined in advance by the alternative. Changing the tail changes the event whose probability is calculated, so it can change the numerical p-value and its interpretation. A test with \(H_a:\mu> \mu_0\) asks whether the data provide evidence in the greater-than direction; a test with \(H_a:\mu\ne\mu_0\) asks about departures in either direction. These are different questions, even when based on the same observed data.

A short justification shows how the direction translates into a tail. For known \(\sigma\), the standardized statistic is a strictly increasing function of \(\bar{x}\), since \(\sigma/\sqrt{n}>0\). Therefore, for any observed sample mean \(\bar{x}_{\text{obs}}\),

\bar{x}\ge\bar{x}_{\text{obs}} \quad\Longleftrightarrow\quad \frac{\bar{x}-\mu_0}{\sigma/\sqrt{n}} \ge \frac{\bar{x}_{\text{obs}}-\mu_0}{\sigma/\sqrt{n}} \quad\Longleftrightarrow\quad Z\ge z_{\text{obs}}

So, when larger means support the alternative, sample means at least as large as the observed mean correspond exactly to statistics at least as large as the observed statistic. Reversing the direction gives the lower-tail event for a less-than alternative. For a two-sided alternative, both directions away from the null value must be represented.

Key takeaway: First state the alternative hypothesis. Then determine which direction or directions of the test statistic support that alternative, and include the observed value and all outcomes farther in those directions. Do not select a tail after inspecting the data.

Common Mistake

Wrong reasoning: “The observed statistic is negative, so the p-value must be a left-tail probability,” or “the observation is on the left, so I will use two tails to be cautious.”

Correction: The tail choice comes from \(H_a\), not from the sign of the observed statistic or a desire to be cautious. A negative statistic calls for a left-tail event when the alternative points left; with a two-sided alternative, outcomes in both directions count. If the alternative points right, the relevant event is still the right tail, even when the observed statistic is negative.

Check Your Understanding

For each question, focus on the event of outcomes at least as extreme as the observed statistic.

If \(H_a:\mu>\mu_0\) and \(t_{\text{obs}}=1.6\), which inequality describes the extreme outcomes?
If \(H_a:\mu<\mu_0\) and \(z_{\text{obs}}=-2.1\), which tail is relevant?
A test has \(H_a:\mu\ne\mu_0\) and an observed statistic of \(-1.8\). In which two directions do extreme outcomes lie?
Why is the sign of the observed statistic not enough, by itself, to choose a tail?
For a right-tailed alternative, should unusually small statistics be included as outcomes at least as extreme in the direction of the alternative? Explain.