Paired T Tests Versus Signed-Rank Tests

One Set of Differences, Two Questions

When measurements are taken on the same people or matched units, the analysis should preserve that pairing. For example, subtracting each person's before-measurement from their after-measurement produces one difference per person. A paired t test and a Wilcoxon signed-rank test both analyze these differences, but they do not automatically answer the same question.

The paired t test is an inference procedure for the population mean difference. The signed-rank test uses the signs and ranks of the differences to assess a symmetric location. A choice between them is not simply a choice between a “parametric” and “nonparametric” method: it depends on the scientific target and on whether the assumptions for the chosen procedure are reasonable.

Definition: For paired observations, define each difference as \(d_i = \text{second measurement}_i - \text{first measurement}_i\), using the same order for every pair. The sample mean difference is \(\bar{x}_d = \frac{1}{n}\sum_{i=1}^{n} d_i\), and the sample standard deviation of the differences is \(s_d\). Reversing the subtraction order reverses the signs and the direction of the conclusion, so the order must be stated.

Both procedures require the paired structure to be represented by one difference per pair. They also rely on independent pairs: the differences from one person or matched unit should not provide dependent repetitions of the differences from another. Pairing allows measurements within a pair to be related; it does not remove a need for independence across pairs.

What Each Procedure Targets

A paired t test is a one-sample t procedure applied to the differences. If \(\mu_d\) denotes the population mean difference, its null hypothesis is commonly \(H_0:\mu_d=0\). The test statistic is

t = \frac{\bar{x}_d - 0}{s_d/\sqrt{n}}, \qquad df=n-1.

The usual reference distribution is exact when the differences are independent and identically distributed normal observations with a common mean and variance. The procedure can also be a useful approximation for a sufficiently large sample when the differences do not have severe skewness or influential outliers. A small sample with pronounced skewness or an extreme difference gives less reason to trust the usual t reference distribution.

The signed-rank test ranks the absolute nonzero differences, attaches each rank the sign of its difference, and compares the positive and negative rank sums. Its exact null calculation relies on independent differences whose signs are equally likely to be positive or negative conditional on their magnitudes under a distribution symmetric about zero. A location interpretation usually assumes a symmetric distribution about a location value; under that model, the test assesses whether that location is zero. In a symmetric population, the location is also the median and, when the mean exists, the mean. Without symmetry, a signed-rank result is not generally a test of the median alone.

A signed-rank analysis therefore does not become appropriate merely because the data are nonnormal. If the differences are strongly skewed, the symmetry condition may be implausible too. In that situation, the analyst should not treat either procedure as automatically justified; the target and the assumptions need to be made explicit.

Why the Paired t Reference Distribution Works

The exact t result follows from the normal-sample model, not just from independence. If \(D_1,\ldots,D_n\) are independent observations from \(N(\mu_d,\sigma_d^2)\), then the standardized sample mean is normal, and the sample variance provides an independent estimate of the common variance. In particular,

\frac{\sqrt{n}(\bar{x}_d-\mu_d)}{\sigma_d}\sim N(0,1), \qquad \frac{(n-1)s_d^2}{\sigma_d^2}\sim\chi^2_{n-1}.

These two quantities are independent under the normal model. Dividing the first by the square root of the second divided by its degrees of freedom gives

\frac{\bar{x}_d-\mu_d}{s_d/\sqrt{n}} = \frac{\sqrt{n}(\bar{x}_d-\mu_d)/\sigma_d} {\sqrt{\{(n-1)s_d^2/\sigma_d^2\}/(n-1)}} \sim t_{n-1}.

This derivation explains why independent normal differences with unequal variances are not enough for the usual exact one-sample t result: the common-variance sample-variance result above would no longer apply.

Worked Comparison on the Same Paired Data

Worked Example: A Small Paired Change Study

Suppose, hypothetically, that eight participants have a symptom score measured before and after a program. Define each difference as after minus before, in score points. The observed differences are \(2, 6, 6, 6, 14, 14, 14, 18\). Positive values mean the score increased. For illustration, suppose the pairs were independently sampled; the small sample does not by itself establish either method's distributional assumptions.

Paired t calculation. The sum of the differences is \(80\), so \(\bar{x}_d=80/8=10\) score points. Their deviations from the mean are \(-8,-4,-4,-4,4,4,4,8\), giving a sum of squared deviations of \(224\). Thus

s_d^2=\frac{224}{7}=32,\qquad s_d=\sqrt{32}=5.657,\qquad SE=\frac{5.657}{\sqrt{8}}=2.000.

For \(H_0:\mu_d=0\), the statistic is \(t=10/2=5.000\), with \(df=7\). Under the exact normal-differences model, the two-sided p-value is \(2P(T_7\geq 5.000)=0.001565\). This is evidence against a zero population mean difference if that model and the sampling assumptions are appropriate; it is not the probability that the null hypothesis is true.

Signed-rank calculation. The absolute differences are \(2\), three values of \(6\), three values of \(14\), and \(18\). Their ranks are \(1\), three average ranks of \(3\), three average ranks of \(6\), and \(8\), respectively. All differences are positive, so \(W_+=1+3(3)+3(6)+8=36\), the maximum possible rank sum. Under the exact sign-symmetry null, each of the \(2^8=256\) sign assignments is equally likely conditional on these magnitudes. Only the all-positive and all-negative assignments are at least this extreme in a two-sided test, giving \(p=2/256=0.007813\).

The tests use the same paired observations but their model-based p-values need not match: the t calculation is about the mean, and the signed-rank calculation is about a symmetric location. This particular sample has repeated values and is very small, so the normal model for an exact t result is not a convincing description of the observed differences. The signed-rank result also depends on a symmetry assumption that cannot be established from eight observations. The arithmetic illustrates the procedures; it does not remove the need to assess their assumptions.

How Outliers and Shape Affect the Choice

An extreme difference can substantially move the mean and increase the sample standard deviation, changing both the paired t statistic and its standard error. In the signed-rank procedure, the size of a difference affects its rank rather than contributing its full numerical magnitude. A very large absolute difference can therefore have less leverage on the rank calculation than on the mean. This relative resistance to extreme magnitudes does not make the signed-rank test assumption-free: symmetry of the differences remains important for its usual location interpretation and exact reference distribution.

Worked Example: A Large Difference and a Different Mean

Consider hypothetical paired score changes, in points, of \(1,2,2,3,3,4,4,25\), again defined as after minus before. Suppose the pairs are independent. The differences are strongly right-skewed in this small sample, so neither an exact normal-differences model nor a symmetric-differences model is well supported by the observed shape.

The sum is \(44\), giving \(\bar{x}_d=5.5\). The squared deviations from \(5.5\) sum to \(442\), so \(s_d^2=442/7=63.14\), \(s_d=7.946\), and \(SE=7.946/\sqrt{8}=2.809\). The paired t statistic is \(5.5/2.809=1.958\), with \(df=7\); its two-sided t-reference p-value is approximately \(0.091\), if that reference model is used. The value \(25\) raises the mean and contributes \(19.5^2=380.25\) to the squared deviations.

For signed ranks, the absolute values \(1,2,2,3,3,4,4,25\) receive ranks \(1\), average ranks \(2.5\), average ranks \(4.5\), average ranks \(6.5\), and \(8\). Since every difference is positive, \(W_+=36\), and the conditional two-sided sign-symmetry calculation is again \(2/256=0.007813\). The ranks limit the influence of the numerical size \(25\), but the strongly skewed sample makes a symmetric-location interpretation questionable. It would be misleading to select the signed-rank test only because it gives the smaller p-value.

The example also shows why the target matters. If the scientific question concerns the average score change, the mean difference is the target even when an outlier makes inference difficult. A rank-based procedure does not answer the mean question just because it is less affected by that outlier.

When the Differences Are Roughly Symmetric

A symmetric shape with no dominating outlier can make both methods reasonable candidates, while still leaving them with different null hypotheses and sampling distributions. A small observed difference between their conclusions is not guaranteed, since the two procedures summarize the data differently.

Worked Example: Symmetric Differences With No Mean Change

Suppose eight independent hypothetical pairs have differences, in minutes, of \(-4,-3,-2,-1,1,2,3,4\). The values are symmetric about zero. Their mean is zero, and the sum of squared deviations is \(60\). Hence \(s_d=\sqrt{60/7}=2.928\), \(SE=2.928/\sqrt{8}=1.035\), and the paired t statistic for \(H_0:\mu_d=0\) is \(t=0/1.035=0\), with \(df=7\). Its two-sided p-value is \(1.000\).

For signed ranks, the positive differences have ranks \(5,6,7,8\), so \(W_+=26\); the negative ranks sum to \(1+2+3+4=10\). The total rank sum is \(36\), and its null center is \(18\). An exact two-sided calculation counts sign assignments with \(W_+\leq10\) or \(W_+\geq26\). There are \(40\) assignments in each tail among \(256\), so \(p=80/256=0.3125\). Both results provide little evidence against their respective zero-location nulls. They are not required to have identical p-values.

A Practical Decision Process

Before calculating either test, state what quantity the study is intended to estimate or test. Then examine the differences themselves—not the two columns of paired measurements separately—because those differences are the input to both procedures.

Define the difference and target.
Specify the subtraction order and units. Decide whether the question concerns a population mean difference or a symmetric location difference.

Check the paired design and independence.
There should be one difference per pair, and different pairs should be independent or reasonably modeled as such.

Assess shape and outliers.
For a paired t test, consider whether the differences are plausibly normal for exact small-sample inference, or whether a large-sample approximation is reasonable. For a signed-rank test, consider whether symmetry is plausible.

Choose and interpret the matching procedure.
Use the paired t procedure for a mean target when its assumptions are defensible; use signed ranks for a symmetric-location target when symmetry is defensible. Report the method's target, assumptions, and result in context.

Sample size matters because a small sample offers limited information about the shape of the difference distribution, while a large sample can make t inference more robust to moderate nonnormality. Neither point is a mechanical rule: severe skewness, extreme outliers, dependence, or a mismatch between the target and procedure still matters. A graph of the differences, numerical summaries, and knowledge of how the pairs were selected all help assess whether an analysis is credible.

Common Mistakes

Assuming paired t means no assumptions. Pairing determines how to form the differences; it does not guarantee independent normal differences or a valid large-sample approximation.
Calling signed ranks a test of the median in every situation. Its standard location interpretation requires symmetry. Without symmetry, a signed-rank result is not generally a test of the median alone.
Choosing by whichever p-value is smaller. The procedures target different features and rely on different assumptions. A smaller p-value does not establish that its assumptions hold or that its target answers the scientific question.
Treating ranks as immune to outliers. Ranking limits the role of extreme magnitudes, but does not eliminate the symmetry requirement or guarantee robustness to every problematic data pattern.

Key takeaway: Paired t and signed-rank procedures both begin with paired differences, but they are not interchangeable. Choose the procedure by identifying the target—mean or symmetric location—and assessing the assumptions for the differences.

Check Your Understanding

Answer in terms of paired differences and the target of each procedure.

A researcher records blood pressure before and after an intervention for each participant. What must be computed before either paired procedure is applied, and why must the subtraction order be stated?
State the condition under which the usual one-sample t reference distribution is exact for a paired t test.
Why does a symmetric-differences assumption matter for interpreting the signed-rank test as a location test?
A single unusually large difference strongly changes the mean but has a relatively limited rank. Does that fact alone make signed ranks the correct procedure? Explain.
If the scientific target is the population mean difference but the differences are strongly skewed in a small sample, what issue remains even if the signed-rank test gives a small p-value?