The Paired Sign Test

Why Analyze Paired Changes?

When the same people are measured before and after a treatment, or when each subject is deliberately matched with another, the two observations in a pair are related. The question is often whether the typical within-pair change is zero. An analysis that treats the before and after measurements as unrelated would ignore the pairing. The paired sign test instead reduces each pair to the direction of its change.

This makes the test useful when differences are not well described by a normal model or when a few unusually large changes could dominate a magnitude-based summary. Its simplicity has a cost: it uses whether a difference is positive or negative, not how large that difference is.

Definition: For pair \(i\), define the paired difference \(d_i\) as the second measurement minus the first measurement. In a before-and-after setting, a common choice is \(d_i=\text{after}_i-\text{before}_i\), so a positive difference indicates an increase. The paired sign test uses the signs of the \(d_i\), omitting differences equal to zero.

Hypotheses and the Sign Count

Let \(m\) be the number of nonzero paired differences, and let \(S_+\) be the number of those differences that are positive. The paired sign test evaluates whether positive and negative nonzero differences are equally likely. In the usual continuous-difference setting, that equality corresponds to a median paired difference of zero.

The continuity qualification matters. If the distribution of differences has a substantial probability of being exactly zero, a zero median alone does not necessarily imply equal probabilities of positive and negative differences after zeros are removed. The sign test then directly tests balance of the signs among nonzero differences; interpreting that balance as a test of a zero median requires care.

Model and hypotheses: Assume the pairs are independent of one another, the pairing is meaningful, and the direction of each nonzero difference has probability one-half of being positive under the null. Then, conditional on the number \(m\) of nonzero differences, \(S_+\sim\operatorname{Binomial}(m,0.5)\) under \(H_0\). For a continuous distribution of paired differences, the hypotheses can be stated as \(H_0:\) median paired difference \(=0\), against one of \(H_a:\) median paired difference \(\ne 0\), \(H_a:\) median paired difference \(>0\), or \(H_a:\) median paired difference \(<0\).

The direction of the alternative must agree with the definition of the differences. If \(d_i=\text{after}_i-\text{before}_i\), a positive alternative asks whether the median change is an increase. Reversing the subtraction reverses the signs and the interpretation.

Why the Binomial Model Applies

Under the null, each nonzero pair contributes either a positive or a negative sign, each with probability \(0.5\). If pairs are independent, these sign outcomes are independent across pairs. Counting the positive signs is therefore equivalent to counting successes in \(m\) independent trials, each with success probability \(0.5\). This is the binomial model for \(S_+\).

P(S_+=k\mid H_0)=\binom{m}{k}(0.5)^k(0.5)^{m-k} =\binom{m}{k}(0.5)^m

This justification also shows why zeros are excluded from \(m\): they are neither positive nor negative and do not contribute a sign trial. A zero difference does not count as evidence in either direction. The test statistic is \(S_+\), together with the number \(m\) of nonzero pairs used to calculate its null distribution.

For an upper-tailed alternative, unusually large values of \(S_+\) support an increase. For a lower-tailed alternative, unusually small values support a decrease. For a two-sided alternative, values far from the null center \(m/2\), in either direction, count as evidence. A p-value is the probability, assuming the null model, of obtaining a sign count at least as extreme as the observed count in the direction or directions specified by \(H_a\).

Applying the Paired Sign Test

Worked Example: A Two-Sided Test of Before-and-After Scores

In a hypothetical exercise program, eight participants complete a score before and after the program. Define each difference as after minus before. The before scores are 12, 10, 14, 9, 11, 13, 8, and 15; the after scores are 14, 11, 16, 10, 12, 14, 9, and 14. Test at \(\alpha=0.05\) whether the median paired change differs from zero.

Check the setup. Each after score is paired with the same participant's before score. Suppose the eight participants can reasonably be treated as independent of one another. There are no zero differences, and the sign test's median interpretation assumes a continuous distribution of changes (or, more generally, balanced positive and negative signs under the null).

Calculate the differences. Subtract before from after, giving \(2,1,2,1,1,1,1,-1\). Thus \(m=8\) and \(S_+=7\). The hypotheses are \(H_0:\) median change \(=0\) and \(H_a:\) median change \(\ne 0\).

Under \(H_0\), \(S_+\sim\operatorname{Binomial}(8,0.5)\). The observed count of seven positives is one sign away from eight, so for a two-sided test we include counts \(7\) and \(8\), and their equally extreme lower-tail counterparts \(1\) and \(0\):

p=P(S_+\ge 7)+P(S_+\le 1) =\frac{\binom{8}{7}+\binom{8}{8}+\binom{8}{1}+\binom{8}{0}}{2^8} =\frac{8+1+8+1}{256} =0.0703

Because \(0.0703>0.05\), we do not reject \(H_0\). These hypothetical data do not provide sufficient evidence at the 5% significance level that the median paired score change differs from zero. This is not proof that the median change is zero.

Worked Example: Testing for an Increase While Omitting Zeros

In a hypothetical before-and-after assessment of daily practice time, 12 students have paired differences, in minutes, of \(2,1,0,3,1,2,-1,1,4,2,1,0\), calculated as after minus before. Test at \(\alpha=0.05\) whether the median change is positive.

Check the setup. Each difference comes from one student's two measurements, and we assume the students' pairs are independent. The two zero differences are ties: they show no change and provide no positive or negative sign. To interpret the test as one about a median under the null, we also require the usual continuous-difference condition or the corresponding sign-balance condition.

After excluding the two zeros, there are \(m=10\) nonzero differences: nine are positive and one is negative. Therefore \(S_+=9\). The hypotheses are \(H_0:\) median change \(=0\) and \(H_a:\) median change \(>0\).

For this upper-tailed alternative, only sign counts of nine or ten are at least as favorable to an increase as the observed count:

p=P(S_+\ge 9) =\frac{\binom{10}{9}+\binom{10}{10}}{2^{10}} =\frac{10+1}{1024} =0.0107

Since \(0.0107<0.05\), we reject \(H_0\) in favor of the positive alternative. The hypothetical data provide evidence that the median paired change is an increase in daily practice time. The units are minutes, but the sign test does not estimate how many minutes the median change is.

Worked Example: Equal Numbers of Positive and Negative Changes

In a hypothetical before-and-after measurement of a laboratory reading, eight paired differences (after minus before) are \(8,1,0.2,-7,-1,-0.5,0,0\) units. Test at \(\alpha=0.05\) whether the median difference differs from zero.

Check the setup. Assume the eight pairs are independent and that the sign-balance model is appropriate under the null. Two pairs have zero differences and are excluded. Among the remaining six, \(m=6\) and \(S_+=3\): three differences are positive and three are negative.

The hypotheses are \(H_0:\) median difference \(=0\) and \(H_a:\) median difference \(\ne 0\). The observed sign count is exactly at the null center, \(m/2=3\). Every possible value of \(S_+\) is at least as extreme as this central value for the two-sided criterion, so:

p=P(S_+\le 3)+P(S_+\ge 3)-P(S_+=3)=1

There is no evidence against \(H_0\) in these signs. Notice that the positive differences include \(8\), while the negative differences include \(-7\). Their magnitudes do not alter the sign count. This example illustrates both the robustness and the information loss of the test.

What the Test Does and Does Not Say

The paired sign test is resistant to extreme difference magnitudes because changing a difference from \(1\) to \(100\) does not change its sign. It is also easy to apply when differences are skewed or when a magnitude-based method's distributional assumptions are doubtful. Its evidence is about direction, not the size of the change.

The test's validity depends on the design and model, not merely on the arithmetic. Different pairs should be independent; repeated observations within a pair are expected and are precisely why the data are paired. The chosen subtraction must match the scientific question. If pairs are clustered, or if dependence across pairs is substantial, the simple binomial model may not be justified.

Also distinguish statistical evidence from practical importance. A small p-value for an increase does not tell us whether the increase is large enough to matter. Conversely, a large p-value can occur because the sample is small or the signs are mixed; it does not establish that no change exists. Since magnitudes are discarded, the paired sign test cannot quantify the typical change in the original measurement units.

Common Mistakes

Counting zeros as negative or positive. A zero difference has neither sign. Exclude it and reduce \(m\) accordingly.
Using the number of pairs as the binomial sample size after ties occur. The binomial count uses \(m\), the number of nonzero differences, not the total number of pairs.
Ignoring the pairing. First calculate one difference within each pair. Do not count all before measurements and after measurements as separate, independent sign observations.
Claiming the test uses the sizes of changes. It uses only signs. Large and small positive differences each contribute one positive sign.
Reading a nonsignificant result as proof of no change. Failure to reject means the signs do not provide sufficient evidence against the null at the selected level; it does not prove the null.

Key takeaway: Choose a consistent within-pair difference, omit zeros, and count positive signs among the remaining \(m\) pairs. Under the null sign-balance model, \(S_+\sim\operatorname{Binomial}(m,0.5)\); the alternative determines which sign counts are considered evidence.

Check Your Understanding

Answer each question using the paired sign test framework.

For before-and-after measurements, what does a positive difference mean if differences are defined as after minus before?
A study has 15 pairs, of which three differences are zero. What value of \(m\) is used for the sign count, and why?
If the alternative is that the median paired difference is less than zero, which values of \(S_+\) provide evidence against the null?
Why does an extremely large positive difference count no more than a small positive difference in this test?
What additional concern arises when the distribution of paired differences has substantial probability exactly at zero?

What You'll Learn