Why Procedure Choice Starts With the Study Design
Sign and rank procedures can be useful when a normal-model test is not a good fit, but they are not interchangeable. The first question is whether the observations are naturally paired. The next is what the data can reasonably support: signs alone, signed magnitudes under a symmetry assumption, or ranks from two independent samples. Choosing a test because it is familiar, without checking these features, can change the question being answered.
The sign test and signed-rank test apply to one sample of differences, often formed from matched pairs or before-and-after measurements. The rank-sum test applies to two independent samples. For any of them, the research question and alternative direction should be set before looking at the test statistic. Exact calculations are often practical for small samples; a normal approximation is an option for sufficiently large samples when its conditions and any tie adjustments are handled appropriately.
A Practical Choice-and-Check Sequence
Use the following sequence before calculating a test statistic. The checks are substantive: they determine whether the test matches the sampling design and whether its reference distribution is appropriate.
If each observation in one condition is matched to a particular observation in another, form one difference per pair. If the groups consist of distinct, independently sampled units, use an independent-sample procedure. Do not treat paired measurements as independent samples.
The sign test uses direction and does not require symmetric differences. The signed-rank test uses direction and magnitude ranks; its usual location interpretation relies on a symmetric distribution of differences. For two independent groups, use the rank-sum test to compare their distributions; a location-shift interpretation needs comparable distributional shapes.
For paired tests, identify zero differences and state how they are handled. Signed-rank calculations also need a rule for tied absolute differences. For rank-sum tests, tied pooled observations receive midranks and affect the variance of the statistic.
For small samples, an exact sign or rank calculation is often feasible and avoids relying on a large-sample approximation. For larger samples, a normal approximation may be suitable, with the appropriate variance adjustment for ties and any continuity correction specified by the method.
What Each Test Assumes and Answers
| Procedure | Design and information used | Key checks and interpretation |
|---|---|---|
| Sign test | One sample of paired differences; positive or negative signs | Zero differences are omitted. Under the null, nonzero signs are equally likely. Under continuity, this tests a zero median of differences without requiring symmetry. |
| Signed-rank test | One sample of paired differences; signs and ranks of absolute differences | Check symmetry about the hypothesized center for the usual location interpretation. Omit zero differences and assign average ranks to tied absolute differences. |
| Rank-sum test | Two independent samples; ranks of pooled observations | Under identical distributions, group labels are exchangeable under the null. A location-shift interpretation additionally requires similar distributional shapes and spread. |
The sign test targets balance between positive and negative nonzero differences. With continuous differences, that balance corresponds to a median difference of zero. The signed-rank test uses more information, but its usual interpretation is more demanding: symmetry of the differences matters. The rank-sum test concerns independent groups. It does not, by itself, establish that a detected difference is specifically a difference in medians if the group distributions have different shapes or spreads.
These are not tests of every possible difference between groups in the same sense. For example, unequal distributions can have the same median, and an apparent rank-sum difference can reflect shape as well as location. State the null and alternative in terms of the assumptions that make the chosen procedure meaningful.
Why the Sign Test Has a Binomial Reference Distribution
Suppose \(n\) paired differences are nonzero and independent, and under the null hypothesis each has probability \(1/2\) of being positive. Let \(B\) be the number of positive differences. Each sign is then a Bernoulli outcome with success probability \(1/2\), so the sum of the \(n\) signs has a binomial distribution.
This gives an exact one-sided probability for observing at least \(b\) positive signs when an increase is the alternative. The derivation also explains why zero differences are not counted as positive or negative: the test conditions on the nonzero signs. For a discrete outcome with many zeros, the sign test’s interpretation and handling of zeros should be stated carefully rather than treating zero as one of the two sign categories.
Worked Example: Paired Data With Zeros and No Symmetry Requirement
A hypothetical clinic records a symptom score before and after a program for ten matched participants. Define \(D_i\) as the after score minus the before score; positive values indicate higher scores after the program. The differences are \(1, 2, 3, 4, 5, 6, 7, -2, 0, 0\). Test for a tendency toward higher scores using a one-sided sign test at the 5% level. Assume the participants’ differences are independent across pairs.
Method and checks. The same participants are measured twice, so this is paired data. A sign test is suitable when symmetry of the differences is not justified. There are two zero differences, which are omitted. Among the eight nonzero differences, seven are positive and one is negative.
Under the null that positive and negative signs are equally likely, \(B\) has a binomial distribution with \(n=8\) and success probability \(1/2\). The observed count is \(b=7\), so the one-sided exact probability is
To four decimal places, the p-value is \(0.0352\), below \(0.05\). Reject the null of balanced signs in favor of a tendency toward positive differences. The conclusion concerns direction among nonzero differences; it does not estimate the size of the change. Common mistake avoided: counting the two zeros as negative signs would use the wrong sample size and would not follow the sign-test procedure.
Worked Example: When Symmetry Supports the Signed-Rank Test
A hypothetical engineering team compares a sensor’s readings before and after recalibration on six matched trials. Let \(D_i\) be the recalibrated reading minus the original reading. The differences are \(+2,+3,+4,+5,+6,-1\). Assume the differences are independent across trials, have no zero values, and are plausibly drawn from a continuous distribution symmetric about the null center of zero. Test for an increase using the exact one-sided signed-rank test at the 5% level.
Method and checks. The readings are paired, and the stated symmetry assumption supports using signs and magnitudes. The absolute differences are \(1,2,3,4,5,6\), so their ranks are \(1,2,3,4,5,6\). The negative difference has rank 1. Therefore the sum of positive ranks is
Under the null, each rank is equally likely to receive a positive or negative sign. The total rank sum is \(1+2+3+4+5+6=21\). The event \(W^+ \ge 20\) is equivalent to the ranks assigned negative signs summing to at most \(1\). There are two such subsets of ranks: the empty subset and the subset containing rank 1. There are \(2^6=64\) sign assignments in total, giving
The exact p-value is \(0.0313\) to four decimal places, so reject the null in favor of an increase at the 5% level. The signed-rank result uses the relative magnitudes as well as their directions. If symmetry were doubtful, the sign test would be the more cautious paired choice; for these signs its one-sided p-value would be \((\binom{6}{5}+\binom{6}{6})/2^6=7/64=0.1094\). Common mistake avoided: a small signed-rank p-value does not remove the need to check the symmetry assumption.
Worked Example: Independent Samples With Tied Pooled Values
Two independent teams in a hypothetical service study record resolution times, in hours, for four cases each. Team A has \(4,6,8,10\); Team B has \(5,7,9,10\). Assume the cases are independently sampled and test whether Team A tends to have lower resolution times. Use a rank-sum test, and compare an exact calculation with a normal approximation.
Method and checks. The teams contain different cases, so the samples are independent rather than paired. Pool and rank the eight observations. The sorted values are \(4,5,6,7,8,9,10,10\). The tied values 10 occupy ranks 7 and 8, so each receives the average rank \(7.5\). Team A’s rank sum is \(R_A=1+3+5+7.5=16.5\).
For an exact lower-tail test under the null of identical distributions, any four of the eight pooled observations could receive the Team A label. There are \(\binom{8}{4}=70\) equally likely allocations. Count allocations with rank sum at most \(16.5\). If neither tied observation is assigned to A, 13 selections of four singleton ranks have sum at most \(16.5\). If exactly one tied observation is assigned to A, there are two choices of that observation and seven selections of three singleton ranks whose sum is at most \(9\); this contributes \(2(7)=14\). If both tied observations are assigned to A, their ranks already sum to \(15\), and the remaining two ranks make the total exceed \(16.5\). Thus 27 allocations are in the lower tail, and
The exact result does not reject at the 5% level. For comparison, the tie-adjusted null mean and variance are
where \(n_A=n_B=4\), \(N=8\), and the only tie has size 2. Using a half-unit continuity correction for the lower tail gives \(z=(16.5-18+0.5)/\sqrt{83/7}\approx -0.2904\), with normal-tail probability about \(0.3858\). This approximation happens to be close here, but the small sample makes the exact calculation preferable. Common mistake avoided: treating the two tied values as if they had distinct ranks would change the rank sum and omit the tie adjustment to the variance.
Common Traps When Choosing and Checking
- Ignoring pairing. Two measurements on the same unit are not two independent samples. Form one difference per matched unit and choose a paired procedure.
- Using signed ranks without checking symmetry. Signed-rank methods use magnitude ranks, but their usual location interpretation relies on a symmetric difference distribution. If that assumption is not credible, consider the sign test.
- Keeping zeros as ordinary signed observations. A zero has neither a positive nor a negative sign. State how zero differences are handled and use the resulting effective sample size.
- Breaking ties arbitrarily. Equal values receive average ranks. In signed-rank tests, tie checks apply to absolute differences; in rank-sum tests, they apply to pooled observations.
- Using a normal approximation automatically. Small samples often allow exact calculations. When approximating, use the correct null variance, tie adjustment where needed, and the specified continuity correction.
- Calling every rank-sum result a median difference. That interpretation needs comparable distributional shapes; otherwise the result may reflect more than a location shift.
A clear solution names the design, the test, the null and alternative, the assumption checks, and why the chosen reference distribution is appropriate. This decision-making approach is aligned with the SOA VEE Mathematical Statistics syllabus.
Check Your Understanding
For each situation, identify the relevant procedure or calculation and check the assumptions before interpreting the result.
- A matched-device study has 11 pairs: two differences are zero, six are positive, and three are negative. For a one-sided increase, find the exact sign-test p-value. (Answer: \(130/512=0.2539\); use \(n=9\) nonzero differences.)
- Five paired differences are \(+1,-2,+3,-4,+5\). Assuming independent, symmetric differences with no ties or zeros, find \(W^+\) and the exact one-sided p-value for an increase. (Answer: \(W^+=9\), p-value \(13/32=0.4063\).)
- A study compares response times from separate participants randomly assigned to two service methods. Which procedure is appropriate among sign, signed-rank, and rank-sum, and what design feature decides? (Answer: rank-sum; the two samples are independent, not paired.)
- Two independent groups each have three observations. Their pooled values are all distinct, and Group A occupies ranks 1, 4, and 5. For a lower-tail exact rank-sum test, find the p-value by counting equally likely allocations of three ranks from 1 through 6. (Answer: \(10/20=0.5000\).)
- A paired signed-rank analysis has many repeated absolute differences and several zero differences. Name two checks or adjustments needed before interpreting a normal approximation. (Answer: omit zero differences according to the signed-rank procedure and assign average ranks to tied absolute differences; use a tie-adjusted variance for the approximation.)