Two-Sided Sign Tests and Tail Conventions

Why Two-Sided Tail Conventions Matter

A sign test reduces observations to positive and negative signs relative to a hypothesized value. For a two-sided question, evidence against the null can come from either an unusually large number of positive signs or an unusually large number of negative signs. The exact p-value must account for both directions, while respecting the discreteness of the sign count.

This tutorial develops two standard ways to calculate that p-value: doubling the smaller one-sided tail, and adding the probabilities of outcomes no more likely than the observed outcome. Under the sign-test null, these exact conventions give the same answer. Recognizing why they agree helps prevent a common error: doubling the wrong tail.

Definition: Let \(D_1,\ldots,D_N\) be independent observations or paired differences, and let \(m_0\) be the hypothesized location. Discard any differences equal to zero, leaving \(n\) nonzero differences. Let \(K\) be the number greater than \(m_0\), so the number below \(m_0\) is \(n-K\). For a continuous population with median \(m_0\), the two-sided sign-test null is \(P(D_i>m_0)=P(D_i<m_0)=0.5\), and hence \(K\sim\operatorname{Bin}(n,0.5)\). The test assumes independent observations and a valid sign probability of 0.5 under the null.

The two-sided hypotheses are \(H_0:\) the population median is \(m_0\), against \(H_A:\) the population median differs from \(m_0\). The sign test uses only the directions of the nonzero deviations; their sizes do not enter the calculation. If observations equal \(m_0\), they contribute no sign and are omitted from \(n\). In particular, \(n\) is not automatically the original number of observations.

The Two Exact Conventions

Once the effective sample size \(n\) and observed positive count \(k\) are known, the null probability of any count \(j\) is

P(K=j)=\binom{n}{j}\left(\frac12\right)^n =\frac{\binom{n}{j}}{2^n},\qquad j=0,1,\ldots,n.

The smaller of the two observed-direction counts is \(b=\min(k,n-k)\). The exact doubled-tail p-value is twice the probability of a result at least that far into the smaller tail, capped at 1:

p_{\text{2-tail}} =\min\left\{1,\;2P(K\le b)\right\} =\min\left\{1,\;\frac{2}{2^n}\sum_{j=0}^{b}\binom{n}{j}\right\}.

The cap matters when the observation is near the center: a p-value cannot exceed 1. This formula includes the observed count in the tail. It is an exact p-value, not an approximation.

The probability-ordering convention instead adds the null probabilities of all count outcomes whose individual probability is no greater than the probability of the observed count:

p_{\text{ord}} =\sum_{\{j:\,P(K=j)\le P(K=k)\}}P(K=j).

The inequality includes outcomes tied in probability with the observed outcome. For the sign-test null, this convention agrees with the doubled-tail calculation. The binomial probabilities are proportional to \(\binom{n}{j}\), and these coefficients increase toward the center of the distribution and decrease away from it. Therefore the counts with probability no greater than \(P(K=k)\) are exactly the two outer tails beginning at \(b\) and \(n-b\), including their endpoints. Their total probability is \(2P(K\le b)\), except that when the tails meet or overlap at the center, the total is 1.

Why the Conventions Agree

Locate the observed count.
Write down \(k\), the number of positive signs, and \(n-k\), the number of negative signs. The observation is equally extreme in either direction according to \(b=\min(k,n-k)\).

Use symmetry.
Under \(K\sim\operatorname{Bin}(n,0.5)\), \(P(K=j)=P(K=n-j)\). The two tails beyond the observed distance from the center therefore have equal probability.

Sum outcomes by probability.
The binomial point probabilities are unimodal: the coefficients \(\binom{n}{j}\) rise to the center and then fall. Thus outcomes no more likely than the observed count are precisely the same two outer tails used by the doubled-tail method. If they cover all possible counts, the p-value is 1.

This agreement depends on the symmetric binomial null for the sign test. It does not mean that all definitions of a two-sided p-value in every discrete test must agree. When reporting a sign-test result, name the exact convention if the setting or software makes that convention relevant.

Worked Calculations

Worked Example: One Positive Sign Among Eight Differences

A hypothetical study records eight independent changes from a fixed reference level. Assume the changes are continuous under the null, and test whether their population median is zero. One change is positive and seven are negative. Calculate the exact two-sided p-value using both conventions, then assess the result at \(\alpha=0.05\).

Method: doubled smaller tail. There are no zero changes, so \(n=8\), and the observed positive count is \(k=1\). The smaller tail count is \(b=\min(1,7)=1\). Under the null, \(K\sim\operatorname{Bin}(8,0.5)\), so

P(K\le 1) =\frac{\binom80+\binom81}{2^8} =\frac{1+8}{256} =\frac{9}{256}.

Doubling gives \(p=2(9/256)=18/256=0.0703125\), or \(0.0703\) to four decimal places. Since \(0.0703>0.05\), do not reject \(H_0\) at the 5% level.

Check by probability ordering. The outcomes with probability no greater than that of \(k=1\) are \(j=0,1,7,8\). Their probabilities sum to

\frac{\binom80+\binom81+\binom87+\binom88}{256} =\frac{1+8+8+1}{256} =\frac{18}{256} =0.0703\text{ (to four decimals)}.

Both exact conventions agree. The common mistake to avoid is doubling the upper tail merely because the test is two-sided: \(P(K\ge1)=255/256\), and doubling it gives a value greater than 1. The correct procedure doubles the smaller tail, not an arbitrarily chosen tail.

Worked Example: Removing Zero Differences Before Testing

In a hypothetical paired evaluation, 12 participants each have a before-and-after measurement. Two paired differences are exactly zero; among the other ten, eight are positive and two are negative. Test whether the population median paired difference is zero using an exact two-sided sign test at \(\alpha=0.05\).

Method: effective sample size, then doubled tail. The two zero differences are omitted, so \(n=12-2=10\), not 12. The positive count is \(k=8\), the negative count is \(10-8=2\), and \(b=\min(8,2)=2\). The lower-tail probability is

P(K\le2) =\frac{\binom{10}{0}+\binom{10}{1}+\binom{10}{2}}{2^{10}} =\frac{1+10+45}{1024} =\frac{56}{1024}.

Thus \(p=2(56/1024)=112/1024=0.109375\), or \(0.1094\) to four decimal places. Because \(0.1094>0.05\), do not reject the null hypothesis.

Check by probability ordering. Since the observed count is \(k=8\), the outcomes no more likely than it are \(j=0,1,2,8,9,10\). Their binomial coefficients sum to \(1+10+45+45+10+1=112\), giving \(p=112/1024=0.1094\) to four decimal places. Using all 12 participants in the binomial calculation would incorrectly treat the two zero differences as positive or negative signs; the test is based on the ten nonzero differences.

Worked Example: A Result Beyond the 1% Threshold

A hypothetical set of 14 independent observations is compared with a proposed median of 25 units. Two observations equal 25, and among the remaining observations 11 are above 25 and one is below 25. Under the continuous sign-test model, calculate the exact two-sided p-value and decide whether to reject at \(\alpha=0.01\).

Method: doubled smaller tail, verified by probability ordering. Omit the two ties, giving \(n=12\), \(k=11\), and \(b=\min(11,1)=1\). The smaller tail is

P(K\le1) =\frac{\binom{12}{0}+\binom{12}{1}}{2^{12}} =\frac{1+12}{4096} =\frac{13}{4096}.

The exact two-sided p-value is \(2(13/4096)=26/4096=0.00634765625\), or \(0.0063\) to four decimal places. Since \(0.0063<0.01\), reject \(H_0\) at the 1% level.

Under probability ordering, the outcomes no more likely than \(k=11\) are \(j=0,1,11,12\). Their coefficients sum to \(1+12+12+1=26\), so this method also gives \(26/4096=0.0063\) to four decimal places. The conclusion is evidence that the median differs from 25; the test does not, by itself, estimate the median or describe the sizes of the deviations.

Tail Conventions, Reporting, and Common Traps

For the exact sign test under \(H_0:p=0.5\), doubling the smaller tail and probability ordering give the same numerical p-value. The distinction is useful for checking work: one method starts with the observed tail count, while the other starts with the probability of the observed outcome. A correct calculation using either method should agree before rounding.

A different reported number can arise when a procedure uses a different convention, such as a mid-p value. A mid-p calculation gives only half the probability of outcomes tied with the observed extremeness, rather than including their full probability as an exact p-value does. For example, in the first worked example the two boundary outcomes \(K=1\) and \(K=7\) together have probability \(16/256\). Subtracting half of that from the exact value \(18/256\) gives the mid-p value \(10/256=0.0391\) to four decimal places. That is not the exact p-value \(0.0703\); it is a distinct convention and should be labelled as such.

The following checks prevent the most frequent errors:

Do not double a tail without checking its direction. First calculate \(b=\min(k,n-k)\), then use \(P(K\le b)\).
Do not count zero differences. Remove them before identifying \(n\); do not assign them a sign.
Do not omit the observed outcome. Both exact conventions include outcomes tied in probability or extremeness with the observed result.
Do not double and leave a result above 1. The two-sided p-value is capped at 1.
Do not confuse exact and mid-p results. State which convention was used when the distinction is material.

Key takeaway: For a two-sided exact sign test, omit zero differences, set \(n\) to the number remaining, and let \(k\) be the positive-sign count. Then double the smaller binomial tail, capped at 1. Under the sign-test null, probability ordering gives the same answer.

Check Your Understanding

For each problem, use the exact two-sided sign-test convention and give probabilities to four decimal places.

Six nonzero differences contain no positive signs and six negative signs. Find the p-value. (Answer: \(2/64=0.0313\).)
Eight nonzero differences contain six positive signs and two negative signs. Find the p-value using the smaller tail. (Answer: \(2(1+8+28)/256=74/256=0.2891\).)
Ten paired measurements include two zero differences. Of the remaining eight, seven differences are positive and one is negative. Find the p-value. (Answer: \(2(1+8)/256=18/256=0.0703\).)
Twelve nonzero differences include 11 positive signs and one negative sign. Find the probability-ordered p-value and decide at \(\alpha=0.01\). (Answer: \((1+12+12+1)/4096=26/4096=0.0063\); reject at the 1% level.)
Nine nonzero differences include two positive signs and seven negative signs. Find the exact p-value and decide at \(\alpha=0.10\). (Answer: \(2(1+9+36)/512=92/512=0.1797\); do not reject at the 10% level.)

What You'll Learn