Two-Sided P-Values and Symmetry

Why Two-Sided P-Values Need Two Tails

A two-sided test looks for evidence against the null hypothesis in either direction. If a test statistic is far above its null center, that may count as evidence; a value equally far below the center may count as evidence too. The p-value must therefore account for outcomes that are at least as extreme as the observed result on both sides.

For a symmetric null distribution, the two relevant tail areas are equal, so one can often calculate a two-sided p-value by doubling one tail. That shortcut depends on the shape of the null distribution and on how extremeness is defined. It is not a universal rule for every two-sided test.

Definition: For a signed test statistic $T$ whose null distribution is symmetric about $0$, define two-sided extremeness by distance from $0$. For an observed value $t_{\mathrm{obs}}$, the two-sided p-value is $$ P\left(|T|\geq |t_{\mathrm{obs}}|\mid H_0\right). $$ It is the null probability of a result at least as far from the null center as the observed result, in either direction.

The absolute value in this definition is important. If $t_{\mathrm{obs}}=1.8$, values $T\geq 1.8$ are at least as extreme in the positive direction, while values $T\leq -1.8$ are at least as extreme in the negative direction. The event includes both tails. For a discrete distribution, the inequalities include outcomes exactly as far from the center as the observed value.

Why the Doubled-Tail Formula Works

Suppose the null distribution of $T$ is symmetric about $0$. Symmetry means that a value $c$ and its reflection $-c$ have matching probability at corresponding points or intervals. For a positive cutoff $c$, the upper-tail event $T\geq c$ and the lower-tail event $T\leq -c$ consequently have equal probabilities. These events are disjoint, so their probabilities add.

\begin{aligned} P(|T|\geq c\mid H_0) &=P(T\geq c\mid H_0)+P(T\leq -c\mid H_0)\\ &=2P(T\geq c\mid H_0), \qquad c>0. \end{aligned}

Taking $c=|t_{\mathrm{obs}}|$ gives the common calculation rule. For a continuous symmetric distribution with cumulative distribution function $F$, this can also be written as $2[1-F(|t_{\mathrm{obs}}|)]$. The symmetry argument, rather than the fact that the test is called “two-sided,” is what licenses the doubling.

If the distribution is discrete, use the inclusive event $|T|\geq |t_{\mathrm{obs}}|$. Symmetry still makes the two tails equal when the center and the extremeness rule are correctly matched. If $t_{\mathrm{obs}}=0$, the event is the entire distribution and the p-value is $1$; do not double a tail at zero.

Identify the null center.
Write down the value around which the signed statistic is centered under the null, commonly $0$ after subtracting the hypothesized parameter value.

Check the null reference distribution.
Establish that it is symmetric about that center. A normal or t reference distribution centered at zero has this property.

Set the observed distance.
For an observed statistic $t_{\mathrm{obs}}$, use $c=|t_{\mathrm{obs}}|$ when the null center is zero.

Combine both tails.
Add the probability of $T\geq c$ to the probability of $T\leq -c$. Double one tail only after confirming that symmetry makes them equal.

Worked Calculations

Worked Example: A Two-Sided Normal Test for a Mean

A hypothetical sensor produces independent normally distributed readings with known standard deviation $\sigma=12$ units. A sample of $n=36$ readings has mean $\bar{x}=104$. Test $H_0:\mu=100$ using the normal null reference distribution, and calculate the two-sided p-value. The population standard deviation is treated as known.

Method. Under $H_0$, the standardized sample mean has a standard normal distribution: $Z=(\bar{X}-100)/(\sigma/\sqrt{n})\sim N(0,1)$. This follows from the normal sampling model and the known value of $\sigma$.

Step 1: Calculate the standard error.

\frac{\sigma}{\sqrt{n}}=\frac{12}{\sqrt{36}}=\frac{12}{6}=2.

Step 2: Calculate the observed statistic.

z_{\mathrm{obs}}=\frac{104-100}{2}=\frac{4}{2}=2.

Step 3: Include both directions. The standard normal distribution is symmetric about zero, so the p-value is $P(|Z|\geq 2)=2P(Z\geq 2)$. From the standard normal distribution, $P(Z\geq 2)=0.02275$.

p=2(0.02275)=0.04550\approx 0.0455.

The p-value is $0.0455$, to four decimal places. As a check, the two equal tail areas are $0.02275$ each, and their sum is $0.04550$. A common mistake is to report only $0.02275$, which is the one-sided upper-tail probability and omits outcomes at least as far below the null center.

Worked Example: Using a t Reference Distribution

A hypothetical laboratory takes a random sample of $n=5$ measurements from a normally distributed population with unknown variance. The sample mean is $\bar{x}=12$, the sample standard deviation is $s=\sqrt{5}$, and the null hypothesis is $H_0:\mu=10$. Calculate the two-sided p-value using the one-sample t statistic.

Method and assumptions. With an independent random sample from a normal population and unknown population variance, the statistic $T=(\bar{X}-\mu_0)/(S/\sqrt{n})$ has a t distribution with $n-1$ degrees of freedom under $H_0$.

Step 1: Calculate the estimated standard error.

\frac{s}{\sqrt{n}}=\frac{\sqrt{5}}{\sqrt{5}}=1.

Step 2: Calculate the observed t statistic and degrees of freedom.

t_{\mathrm{obs}}=\frac{12-10}{1}=2,\qquad \nu=n-1=4.

Step 3: Calculate both tails. The t distribution with four degrees of freedom is symmetric about zero. A t table or statistical calculator gives $P(T\geq 2)=0.05806$ for $T\sim t_4$. Therefore,

p=P(|T|\geq 2)=2(0.05806)=0.11612\approx 0.1161.

The p-value is $0.1161$, to four decimal places. The check is that the lower-tail probability at or below $-2$ is also $0.05806$, so the total probability in the two extreme tails is $0.11612$. The degrees of freedom matter: using a standard normal tail instead would use the wrong null reference distribution.

Worked Example: Adding Tails for a Symmetric Discrete Null

A hypothetical quality audit inspects $8$ independent items. Under the null model, each item independently has probability $0.5$ of meeting a specified criterion. Let $X$ be the number meeting the criterion, and suppose the observed count is $X=7$. Calculate the two-sided p-value using distance from the null mean as the extremeness rule.

Step 1: Find the null center and the observed distance. Under $H_0$, $X\sim\operatorname{Binomial}(8,0.5)$, so $E[X]=8(0.5)=4$. The observed count is $3$ above the center. Outcomes at least as far away satisfy $|X-4|\geq 3$, which means $X\leq 1$ or $X\geq 7$.

Step 2: Calculate one tail. The upper tail is

\begin{aligned} P(X\geq 7) &=P(X=7)+P(X=8)\\ &=\binom{8}{7}(0.5)^8+\binom{8}{8}(0.5)^8\\ &=\frac{8+1}{256}=\frac{9}{256}. \end{aligned}

Step 3: Use symmetry and report the result. Since $X\sim\operatorname{Binomial}(8,0.5)$ is symmetric about $4$, the lower tail $P(X\leq 1)$ is also $9/256$.

p=\frac{9}{256}+\frac{9}{256} =\frac{18}{256}=0.0703125\approx 0.0703.

The p-value is $0.0703$, to four decimal places. As a check, the two tail regions are disjoint and each contains $9$ of the $256$ equally likely outcomes. Counting just the upper tail would give $9/256$, not the two-sided p-value.

When Doubling Is Not Justified

A null distribution can be asymmetric, and then outcomes equally far from a chosen center need not have equal tail probabilities. In that setting, the two-sided extremeness rule must be stated; there is not one universal rule for ordering every possible discrete outcome. If the rule is distance from the null mean, the p-value is the probability of all outcomes at least that far from the mean. One may not replace the opposite tail by a copy of the observed tail unless symmetry has been established.

Worked Example: An Asymmetric Binomial Null

A hypothetical screening process has independent outcomes with null probability $0.2$ of a positive result. In $5$ screens, the observed number of positives is $X=2$. Use distance from the null mean as the two-sided extremeness rule. Show why doubling the upper tail is incorrect.

Step 1: Find the null center and extremeness event. Under $H_0$, $X\sim\operatorname{Binomial}(5,0.2)$, so $E[X]=5(0.2)=1$. The observed distance from the mean is $|2-1|=1$. Thus, outcomes at least as far from the mean satisfy $|X-1|\geq1$, or $X=0$ or $X\geq2$.

Step 2: Calculate the probability of the event. The needed binomial probabilities are

\begin{aligned} P(X=0)&=(0.8)^5=0.32768,\\ P(X=2)&=\binom{5}{2}(0.2)^2(0.8)^3=0.2048,\\ P(X=3)&=\binom{5}{3}(0.2)^3(0.8)^2=0.0512,\\ P(X=4)&=\binom{5}{4}(0.2)^4(0.8)=0.0064,\\ P(X=5)&=(0.2)^5=0.00032. \end{aligned}

The outcome $X=1$ is the only one excluded. Adding the stated probabilities gives $0.32768+0.2048+0.0512+0.0064+0.00032=0.5904$. Therefore the p-value under the specified distance rule is $0.5904$.

Step 3: Check the doubling shortcut. The upper tail is $P(X\geq2)=0.2048+0.0512+0.0064+0.00032=0.26272$. Doubling it gives $0.52544$, not $0.5904$. The binomial distribution with probability $0.2$ is not symmetric about its mean, and its opposite extreme tail $P(X=0)=0.32768$ is larger than $P(X\geq2)=0.26272$. Thus, doubling fails.

Common Exam Traps

Reporting one tail for a two-sided question. A one-sided tail is not the probability in both directions. For a symmetric null distribution and a positive observed distance, add the matching opposite tail.
Doubling without checking the null distribution. A two-sided alternative does not make the null distribution symmetric. Verify the symmetry of the statistic's null reference distribution before doubling.
Using the wrong center. Symmetry must be about the null center for the statistic being tested. If a statistic is centered at a nonzero null value, measure distance from that value or first recenter it.
Ignoring discreteness. Include outcomes tied with the observation in extremeness, and add the actual lower and upper tail probabilities. A discrete tail need not match a continuous formula mechanically.
Using the wrong reference distribution. A standardized statistic based on an estimated variance may have a t reference distribution under stated assumptions, not a standard normal one.

Key takeaway: A two-sided p-value combines the null probabilities of outcomes at least as extreme in either direction. For a signed statistic with a symmetric null distribution centered at zero, symmetry makes the two tails equal and permits doubling one tail. Without that symmetry—and a clearly specified definition of extremeness—calculate the two tail areas separately.

Check Your Understanding

For each problem, use the stated extremeness rule and give the two-sided p-value. Round probability answers to four decimal places where needed.

Under $H_0$, $Z\sim N(0,1)$ and the observed statistic is $z_{\mathrm{obs}}=-1.25$. Given $\Phi(1.25)=0.8944$, calculate the two-sided p-value. (Answer: $0.2112$)
Under $H_0$, $T\sim t_{16}$ and $t_{\mathrm{obs}}=1.75$. A t-distribution calculator gives $P(T\geq1.75)=0.0496$. Calculate the two-sided p-value. (Answer: $0.0992$)
Under $H_0$, $X\sim\operatorname{Binomial}(10,0.5)$, and the observed count is $X=8$. Use distance from the null mean as extremeness. Calculate the two-sided p-value. (Answer: $0.1094$)
Under $H_0$, $Z\sim N(0,1)$ and $z_{\mathrm{obs}}=2.40$. Given $P(Z\geq2.40)=0.0082$, calculate the two-sided p-value. (Answer: $0.0164$)
Under $H_0$, $X\sim\operatorname{Binomial}(4,0.25)$, and the observed count is $X=2$. Use distance from the null mean as extremeness. Calculate the p-value by summing the probabilities of all outcomes at least as far from the mean as the observed count. (Answer: $0.5781$)

What You'll Learn