Why Paired Tests Need Paired Differences
When the same person is measured before and after an intervention, the two measurements are related. An analysis that treats them as independent observations ignores that pairing. Instead, calculate one difference for each person, using a consistent direction such as after minus before. The sign test and Wilcoxon signed-rank test both analyze these differences, but use different information: the sign test uses direction only, while the signed-rank test also uses the ranks of the difference magnitudes.
The sign test can test whether positive and negative differences are equally likely under its null model. The signed-rank test uses a stronger symmetry assumption to make use of both signs and magnitudes. The choice between them is therefore not merely a choice between two calculations: it depends on the assumptions that are reasonable for the paired differences.
Hypotheses and Exact Test Rules
For a one-sided question about an increase, the sign-test null model assigns probability \(1/2\) to each sign among nonzero differences. If there are \(n\) nonzero differences and \(B\) is the number that are positive, then \(B\sim\operatorname{Binomial}(n,1/2)\) under the null. The alternative is that positive differences are more likely than negative ones. For a decrease, count negative differences instead. For a two-sided test, departures in either direction count as evidence.
This sign-test model does not require a symmetric distribution of difference magnitudes. Its median interpretation is cleanest when the difference distribution is continuous at zero: testing equal probabilities of positive and negative differences then tests a median of zero. With a point mass at zero, the test directly concerns the balance of positive and negative signs among nonzero differences; that balance should not automatically be described as every possible definition of a zero median.
The signed-rank test first removes zero differences, ranks the remaining absolute differences from smallest to largest, and attaches each rank its original sign. Tied absolute differences receive their average rank. Write \(W^+\) for the sum of positive ranks and \(W^-\) for the sum of negative ranks. If the nonzero ranks sum to \(R\), then \(W^++W^-=R\).
The signed-rank symmetry condition matters. Without it, a large difference is not necessarily as likely to be positive as negative under the null, so the exact sign-assignment calculation need not be valid. With zeros, omit them from ranking and report how many were omitted. With tied absolute differences, use average ranks and enumerate the resulting rank values; do not break ties arbitrarily.
A Short Justification of the Exact Signed-Rank Distribution
Under the signed-rank null, changing the sign of any nonzero difference while holding its absolute value fixed leaves the conditional probability unchanged. For each of the \(n\) differences, there are two possible signs, and the null gives each of the \(2^n\) sign patterns probability \(1/2^n\). A particular pattern contributes to \(W^+\) the sum of the ranks assigned positive signs. Therefore, counting the sign patterns whose rank sums are at least as extreme as the observed sum gives the exact tail probability. Repeated rank sums must be counted repeatedly because distinct sign patterns can produce the same sum.
For a two-sided test, this tutorial uses the exact probability of a rank sum at least as far from its null center \(R/2\) as the observed \(W^+\). By symmetry, this is equivalent to doubling the smaller one-sided tail and capping the result at 1. State the tail convention used when reporting an exact p-value, especially when the distribution is discrete.
Worked Paired Tests
Worked Example: A One-Sided Sign Test With a Zero Difference
In a hypothetical clinic review, nine participants have a score recorded before and after a brief program. The after-minus-before differences are \(3,1,4,0,2,5,1,3,-2\). Test at the 5% significance level whether positive changes are more likely than negative changes. Also report the two-sided p-value.
Method and assumptions. Use the sign test. The participant pairs are assumed independent, and under the null each nonzero difference is equally likely to be positive or negative. The one-sided alternative was specified as an increase.
Count signs and set the null distribution. There are seven positive differences, one negative difference, and one zero. Omit the zero, so \(n=8\), and the observed positive count is \(B=7\). Under the null, \(B\sim\operatorname{Binomial}(8,1/2)\).
Calculate the one-sided p-value. Results at least as favorable to the increase alternative are \(B=7\) or \(B=8\):
Since \(0.0352<0.05\), reject the null for the prespecified one-sided test. The data provide evidence that positive changes are more likely than negative changes under the sign-test model.
Calculate the two-sided p-value. The observed count is one away from 8, so the equally extreme opposite tail is \(B\leq1\). By symmetry,
The two-sided test would not reject at 5%. The one-sided conclusion is appropriate only if the increase direction was chosen before examining the observed signs. Common mistake avoided: including the zero in \(n\) would incorrectly use a binomial distribution with nine trials.
Worked Example: An Exact Signed-Rank Test With Distinct Magnitudes
A hypothetical equipment team records five paired differences in operating time, after minus before: \(-1,2,3,4,5\) minutes. Assuming the difference distribution is symmetric under the null, test at the 5% level for a positive location shift. Calculate both the one-sided and two-sided exact p-values.
Method and assumptions. Use the exact signed-rank test. The five pairs are independent, the differences have a symmetric distribution under the null, and there are no zero differences or tied absolute differences.
Rank the magnitudes. The absolute differences are \(1,2,3,4,5\), so their ranks are \(1,2,3,4,5\). The negative difference has rank 1. The positive ranks are 2, 3, 4, and 5, giving
Find the one-sided p-value. Under the null, every subset of ranks has probability \(1/2^5=1/32\). To obtain \(W^+\geq14\), the positive ranks must sum to 14 or 15: the subsets \(\{2,3,4,5\}\) and \(\{1,2,3,4,5\}\). Thus
Find the two-sided p-value. The null center is \(R/2=7.5\). The observed sum is \(14\), which is 6.5 above the center; the equally extreme lower sums are \(W^+\leq1\), namely 0 and 1. There are two assignments in each tail, so
Neither p-value is below 0.05, so neither test rejects at that level. Common mistake avoided: using only the number of positive differences would discard the rank information and would not calculate a signed-rank p-value.
Worked Example: Comparing the Tests With a Zero and Tied Magnitudes
A hypothetical six-person assessment records after-minus-before differences of \(-1,2,-2,4,7,0\). Test for a two-sided change at the 5% level, calculating both exact procedures. For the signed-rank test, assume symmetry under the null.
Sign test. Omit the zero. Of the five remaining differences, three are positive and two are negative. Under the sign-test null, the positive count \(B\) has a \(\operatorname{Binomial}(5,1/2)\) distribution. The observed count 3 is at least as far from 2.5 as every possible count, so the two-sided p-value is 1. Equivalently, doubling the smaller tail gives \(2P(B\leq2)=2(16/32)=1\). There is no evidence of a directional imbalance in signs.
Signed-rank test. Omit the zero and rank the five nonzero absolute differences \(1,2,2,4,7\). The tied values 2 occupy ranks 2 and 3, so each receives average rank \(2.5\). The full set of ranks is \(1,2.5,2.5,4,5\). The positive differences have ranks \(2.5,4,5\), giving
The null center is \(7.5\), so the observed distance from the center is \(4\). To count assignments at least this far from the center, double each rank to avoid fractions. The doubled ranks are \(2,5,5,8,10\), with total 30. The lower tail has doubled sums at most 7. These arise from choosing no ranks (sum 0), rank 2, either of the two rank-5 entries, or rank 2 together with either rank-5 entry: \(1+1+2+2=6\) assignments. By symmetry, there are also 6 assignments in the upper tail. Therefore,
Both p-values exceed 0.05, so neither procedure rejects. The signed-rank calculation uses the relative magnitudes, while the sign test uses only the three positive and two negative signs. Common mistake avoided: assigning the tied observations consecutive ranks 2 and 3 would give one observation a rank advantage that the data do not support.
Choosing and Reporting the Paired Test
| Feature | Sign test | Signed-rank test |
|---|---|---|
| Information used | Positive and negative signs | Signs and ranks of absolute differences |
| Null condition | Equal probabilities of positive and negative nonzero differences | Symmetric difference distribution centered at zero |
| Zeros | Omit from the sign count | Omit before ranking and report the number omitted |
| Tied magnitudes | Do not affect the sign count | Assign average ranks and enumerate sign patterns |
A signed-rank result can be more informative when symmetry is plausible because it incorporates magnitudes through their ranks. If symmetry is not credible, the sign test may be the more defensible choice, although it discards magnitude information. Neither procedure tests a change in the mean merely by virtue of being applied to before-and-after data. State the paired difference direction, the alternative, the test assumptions, the number of zero differences, and the exact tail convention.
A p-value measures how unusual the observed statistic, or a more extreme one, would be under the stated null model. It is not the probability that the null hypothesis is true, and failure to reject does not prove there is no change. These distinctions are important when translating a calculation into a professional conclusion aligned with the SOA VEE Mathematical Statistics syllabus.
Check Your Understanding
For each problem, use the stated exact-test convention and assumptions.
- A paired sign test has six nonzero differences, five positive and one negative. Find the one-sided p-value for an increase and decide whether to reject at 10%. (Answer: \(P(B\geq5)=7/64=0.1094\); do not reject at 10%.)
- Seven nonzero paired differences are all positive. Find the exact two-sided sign-test p-value. (Answer: \(2/2^7=0.0156\).)
- For paired differences \(-1,2,3,4\), assume the signed-rank null. Find the one-sided exact p-value for a positive shift. (Answer: \(W^+=9\), so \(P(W^+\geq9)=2/16=0.1250\).)
- For paired differences \(1,-3,3,6,0\), assume symmetry under the signed-rank null. Find the two-sided exact signed-rank p-value. (Answer: nonzero ranks are \(1,2.5,2.5,4\), \(W^+=7.5\), and \(p=8/16=0.5000\).)
- A signed-rank analysis has several zero differences and one tied pair of absolute differences. State how to handle them in an exact calculation. (Answer: omit zeros before ranking, assign average ranks to tied absolute differences, and enumerate all sign assignments for the remaining nonzero observations.)