From Reaction Times to Paired Differences
A before-and-after study measures the same participants twice. For example, a researcher might record reaction time before and after a brief training session. The question is often whether the average reaction time changed. Because each participant contributes both measurements, the analysis should preserve that pairing rather than treat the before and after values as two unrelated groups.
A paired t test turns each participant’s two measurements into one difference, then tests the population mean of those differences. The choice of subtraction order matters: it determines what a positive or negative mean difference says. Here, reaction time is measured in milliseconds, and a smaller value means a faster response.
If the scientific question is whether the training is associated with a decrease in reaction time, these definitions lead to \(H_0:\mu_D=0\) and \(H_a:\mu_D>0\). The null hypothesis says the population mean before-minus-after difference is zero; the alternative says it is positive. The hypotheses must match the chosen coding and the question of interest.
Worked Example: Testing Mean Reaction-Time Change
Worked Example: Testing Mean Reaction-Time Change
Suppose, hypothetically, that six participants complete the same reaction-time task before and after a training session. The recorded times and differences are:
| Participant | Before (ms) | After (ms) | \(D_i=\text{before}-\text{after}\) (ms) |
|---|---|---|---|
| 1 | 310 | 302 | 8 |
| 2 | 298 | 294 | 4 |
| 3 | 325 | 318 | 7 |
| 4 | 305 | 302 | 3 |
| 5 | 315 | 309 | 6 |
| 6 | 300 | 298 | 2 |
Check the design and conditions. The same six people were measured twice, so the observations are paired. The six participants are assumed to represent independent observational units; one person’s difference should not determine another person’s difference. For a small-sample paired t procedure, the distribution of population differences should be approximately normal, with no severe outliers in the observed differences. Here the differences range from 2 to 8 ms and have no isolated extreme value. A plot of the six differences would still be useful, but with only six observations it cannot establish normality reliably. The example assumes the differences are reasonably consistent with a roughly symmetric distribution. If these participants were not randomly sampled, conclusions should be limited to the group and process they represent.
Calculate the sample summaries. The differences are \(8,4,7,3,6,2\) ms. Their mean is
The deviations from 5 are \(3,-1,2,-2,1,-3\), and their squared values sum to \(28\). Thus the sample standard deviation and standard error are
Set up and calculate the test. Let \(\mu_D\) denote the population mean before-minus-after reaction-time difference. To test whether the mean reaction time decreased, use \(H_0:\mu_D=0\) against \(H_a:\mu_D>0\). At significance level \(\alpha=0.05\), the test statistic is
Under the null model and the paired t-test conditions, this statistic is compared with a t distribution with 5 degrees of freedom. The one-sided upper-tail p-value is approximately 0.00177. Since \(0.00177<0.05\), reject \(H_0\) at the 5% significance level.
State the conclusion in context. These hypothetical data provide strong evidence that the population mean before-minus-after reaction-time difference is positive. Equivalently, they provide evidence that mean reaction time after the session is lower than before it, for the population represented by the study’s sampling process. The estimated mean decrease is 5.000 ms. This test does not by itself establish that training caused the decrease: practice, fatigue, time, or other changes could also contribute in a before-and-after design.
Why the Paired t Statistic Has This Form
Once differences have been defined, the paired test is a one-sample t procedure applied to those differences. The target is the mean change \(\mu_D\), the estimate is \(\bar{d}\), and the sample standard deviation \(s_D\) measures variation among participants’ changes. If the differences are independent and normally distributed, then under \(H_0:\mu_D=0\), the standardized sample mean follows a t distribution with \(n-1\) degrees of freedom. For sufficiently well-behaved differences, the t procedure is also commonly used as an approximation.
The test statistic follows by measuring how many estimated standard errors separate the observed mean difference from its null value:
Here \(\mu_{D,0}\) is the value specified by the null hypothesis, usually 0 for a test of no mean change. The denominator is the estimated standard error of \(\bar{d}\). If the null is true and the model conditions hold, dividing by this estimated standard error gives the t reference distribution, with \(n-1\) degrees of freedom because the sample mean is estimated from the same \(n\) differences.
Worked Example: Reversing the Difference Coding
Worked Example: Reversing the Difference Coding
Use the same hypothetical reaction-time measurements, but define \(D_i^*=\text{after}_i-\text{before}_i\). The six differences are now \(-8,-4,-7,-3,-6,-2\) ms. Reversing subtraction changes the signs of all differences, so \(\bar{d}^*=-5.000\) ms and \(s_{D^*}=2.366\) ms; the standard error remains \(0.9661\) ms.
To test for a decrease in reaction time, this coding requires \(H_0:\mu_{D^*}=0\) and \(H_a:\mu_{D^*}<0\), because a decrease makes after-minus-before negative. The statistic is
The lower-tail p-value for \(t=-5.175\) is approximately 0.00177, the same as the upper-tail p-value for \(t=5.175\) in the first coding. The conclusion is unchanged: there is evidence of a decrease. The estimated mean change under this coding is \(-5.000\) ms. A coding choice is not a statistical result; it is a sign convention that must be carried through consistently in the hypotheses, statistic, tail, and interpretation.
Reading Paired-Test Software Output
Statistical software typically reports the sample mean difference, the test statistic, degrees of freedom, and a p-value. It may also label the comparison as “before minus after” or “after minus before.” Check that label before interpreting the sign. A value of \(t=5.175\) means something different from \(t=-5.175\) only when the subtraction order and alternative are not also considered.
Worked Example: Translating Software Output Into a Conclusion
For the six reaction-time pairs above, suppose software is instructed to calculate before minus after and test whether the mean difference is greater than 0. It reports the following rounded output:
| Output item | Reported value |
|---|---|
| Mean difference (before minus after) | 5.000 ms |
| Standard error | 0.9661 ms |
| t statistic | 5.175 |
| Degrees of freedom | 5 |
| One-sided p-value (greater than 0) | 0.00177 |
The software’s values agree with the hand calculations: the estimate is \(5.000\) ms and the statistic is \((5.000-0)/0.9661=5.175\). The p-value is the probability, assuming \(\mu_D=0\) and the t-test model applies, of obtaining a t statistic at least as large as 5.175 in the specified upper-tail direction. It is not the probability that the null hypothesis is true.
Because \(0.00177<0.05\), the result is statistically significant at \(\alpha=0.05\). In context, the data give evidence that the population mean before-minus-after change is positive, corresponding to lower mean reaction time after the session. The test estimates the size of the mean change as 5.000 ms; the p-value addresses how compatible the observed statistic is with the null model, not whether a 5 ms difference is practically important.
Common Mistakes and Why the Checks Matter
- Using the wrong subtraction order. A positive mean difference does not inherently mean improvement. It means improvement only if the coding makes improvement positive, as before minus after does for a time that is expected to decrease.
- Treating the two measurement occasions as independent groups. The paired procedure uses one difference per participant. The number of observations for the t test is the number of complete pairs, not twice that number.
- Checking the separate before and after distributions instead of the differences. The paired t procedure’s normality condition concerns the distribution of \(D_i\), not the two marginal distributions separately. A plot of the differences and a check for extreme values are relevant diagnostics.
- Reading a one-sided p-value as two-sided, or vice versa. The alternative hypothesis determines which tail is used. A p-value must correspond to the direction stated before interpreting the result.
- Claiming the session caused the change from a before-and-after comparison alone. Pairing controls for stable differences between participants, but it does not eliminate time-related changes or other explanations. Causal conclusions require a design that supports them.
- Misstating a non-significant result. Failing to reject \(H_0\) would mean the data do not provide sufficiently strong evidence against it at the chosen \(\alpha\); it would not prove that the mean change is exactly zero.
The paired t test is useful when the scientific question concerns average within-person change and the measurements are naturally linked. Its strength comes from analyzing those changes directly. Its reliability depends on the pairing being correctly recorded, the differences being appropriate for t-based inference, and the conclusion staying within what the design can support.
Check Your Understanding
Use the difference definition and conditions carefully when answering.
- If reaction time is coded as after minus before, what sign would indicate a decrease in reaction time?
- For \(n=12\) complete pairs, what are the degrees of freedom for a paired t test, and what data values determine them?
- Why should a diagnostic plot focus on the paired differences rather than only on the before and after measurements separately?
- A software output reports \(t=-2.4\) for after minus before. What additional information is needed before deciding whether this supports a decrease in reaction time?
- If a paired test has \(p=0.12\) at \(\alpha=0.05\), what is a correct conclusion, and what would be an incorrect claim?