Why Software Output Needs Careful Reading
Paired t test software can summarize a comparison in a few numbers, but those numbers are meaningful only when the pairing and the direction of subtraction are clear. A negative mean difference might indicate a decrease, or it might simply reflect that the software subtracted the measurements in the reverse order from the one you expected. The output also contains related quantities—the standard error, t statistic, degrees of freedom, p-value, and confidence interval—that answer different questions.
This tutorial focuses on reading those fields and translating them into a conclusion in context. The analysis concerns the differences within pairs, not two unrelated sets of measurements. Before interpreting the output, confirm how the software defined each difference and whether the paired t conditions are reasonable.
What the Output Fields Mean
Software layouts differ, but the important quantities have standard meanings. A report may call \(\bar{d}\) “mean difference,” “paired difference,” or “estimate.” The order of the measurements might be shown as “before − after,” “group 1 − group 2,” or a similar label. Read that label before interpreting the sign: a positive result means the first quantity in the stated subtraction is larger on average, while a negative result means it is smaller.
- Mean difference, \(\bar{d}\): the average of the observed within-pair differences, in the original measurement units. It describes the estimated direction and size of the average change or contrast in the sample.
- Standard error, \(SE\): the estimated standard deviation of the sampling distribution of \(\bar{d}\). For a paired t test, \(SE=s_d/\sqrt{n}\), where \(n\) is the number of complete pairs.
- t statistic: the number of standard errors between the observed mean difference and the null value. For a null value of zero, \(t=\bar{d}/SE\). Its sign follows the sign of the difference under the software’s subtraction order.
- Degrees of freedom, \(df\): for the ordinary paired t test, \(df=n-1\). Thus, 12 complete pairs give 11 degrees of freedom.
- p-value: the probability, assuming the null hypothesis and the test model, of obtaining a t statistic at least as extreme as the observed statistic in the direction or directions specified by \(H_a\). A one-sided and a two-sided test can therefore give different p-values for the same t statistic.
- Confidence interval: an interval estimate for \(\mu_d\), expressed in the same units as the differences. Its endpoints are interpreted using the stated confidence level and the population mean difference, not as a range containing most individual differences.
The model behind the calculation is the one-sample t model applied to the \(n\) paired differences. The pairs should be independent of one another, and the differences should be quantitative. For a small sample, the distribution of the differences should be reasonably close to normal, without severe outliers; with a sufficiently large sample, the t procedure is generally more robust to moderate departures from normality. These conditions concern the differences, not the separate before and after measurements considered independently.
How the Reported Numbers Fit Together
The software’s t statistic is not a separate effect measure. It standardizes the observed mean difference relative to its estimated sampling variability. If the null mean difference is \(\delta_0\), the statistic is
For the common null value \(\delta_0=0\), a mean difference of \(-1.80\) with a standard error of \(0.693\) gives a t statistic near \(-2.60\). The numerator gives the observed departure from the null, while the denominator measures its estimated sampling variability. Under the null hypothesis and the paired t model, this standardized statistic follows a t distribution with \(n-1\) degrees of freedom.
A confidence interval for the mean difference is commonly reported as
where \(t^*\) is the appropriate t critical value for the confidence level and degrees of freedom. This formula explains why the interval is centered at the estimated mean difference and why its width depends on the standard error. A full study of confidence intervals for paired differences develops their construction and interpretation further; here, the practical task is to read the reported endpoints and explain what they estimate.
Confirm what one pair represents, how many complete pairs were analyzed, and which measurement was subtracted from which.
State the mean difference in context and units, then identify its standard error.
Check the null and alternative hypotheses, then interpret the t statistic, degrees of freedom, and p-value for the reported tail.
Describe the confidence interval for the population mean difference and state the evidence relative to the chosen significance level.
Worked Output Interpretations
Worked Example: A Two-Sided Test of Mean Change
In a hypothetical exercise study, 12 participants each have a measurement recorded before and after a program. The software defines each difference as before minus after and reports: mean difference \(=-1.80\) minutes, standard error \(=0.693\) minutes, \(t=-2.60\), \(df=11\), two-sided \(p=0.02469\), and a 95% confidence interval of \([-3.33,-0.275]\) minutes. Assume the 12 participants form independent pairs and that the differences show no severe outliers or strong departure from normality. The hypotheses are \(H_0:\mu_d=0\) and \(H_a:\mu_d\ne0\).
Direction and estimate. Because the subtraction is before minus after, the mean difference of \(-1.80\) minutes means that the after measurements are higher by an average of 1.80 minutes in this sample. This statement follows from the stated subtraction order; it would reverse if the software had used after minus before.
Check the statistic and degrees of freedom. Using the displayed rounded values, \(\bar{d}/SE=-1.80/0.693\approx-2.597\), which rounds to the reported \(t=-2.60\). The 12 complete pairs give \(df=n-1=12-1=11\). Software calculates its p-value using its underlying, unrounded values; the displayed \(t\) is rounded.
Interpret the p-value. Under the null hypothesis of zero population mean difference, and assuming the paired t model, a two-sided p-value of 0.02469 means that a t statistic at least as far from zero as the observed statistic would occur with probability about 0.02469. At significance level \(\alpha=0.05\), this is evidence against \(H_0\). It does not give the probability that \(H_0\) is true.
Interpret the interval. The reported interval estimates the population mean before-minus-after difference. In repeated sampling under the model, the method used to construct 95% intervals would cover the true mean difference in about 95% of such samples. In this example, the interval suggests a mean before-minus-after difference from about \(-3.33\) to \(-0.275\) minutes, or equivalently an average increase after the program of about 0.275 to 3.33 minutes. The interval excludes zero, in agreement with the two-sided test at the 0.05 level.
Worked Example: A One-Sided Test and a Positive Difference
In a hypothetical paired assessment, 10 learners complete a task under each of two conditions. The software defines differences as condition A minus condition B. It reports a mean difference of \(0.900\) points, \(SE=0.300\) points, \(t=3.00\), \(df=9\), one-sided \(p=0.007478\), and a 95% two-sided confidence interval of \([0.221,1.579]\) points. Suppose the question specified in advance is whether condition A has a higher mean score than condition B, so \(H_a:\mu_d>0\). Assume the pairs are independent and the distribution of their differences is suitable for paired t inference.
The positive estimate means that condition A scored higher by an average of 0.900 points in the observed pairs. The reported test statistic checks directly: \(0.900/0.300=3.00\). There are 10 complete pairs, so \(df=10-1=9\).
The one-sided p-value is \(0.007478\), the probability under \(H_0:\mu_d=0\) of a t statistic at least as large as 3.00 in the specified positive direction. Since \(0.007478<0.05\), the result is statistically significant at the 0.05 level in support of a positive mean difference. The p-value should not be doubled here: the stated alternative is one-sided, and the test’s tail was chosen to match that alternative.
The software’s 95% two-sided interval runs from 0.221 to 1.579 points. In context, it estimates a positive population mean difference, with values in that range compatible with this interval procedure at the stated confidence level. This reported two-sided interval is distinct from the one-sided test, even though both results point in a positive direction.
Worked Example: A Small t Statistic and an Interval Including Zero
In a hypothetical paired comparison of two measurement methods, 8 objects are measured by both methods. The software uses method A minus method B and reports a mean difference of \(0.400\) units, \(SE=0.800\) units, \(t=0.500\), \(df=7\), two-sided \(p=0.6325\), and a 95% confidence interval of \([-1.492,2.292]\) units. Assume the objects are independent and the paired differences are reasonably compatible with the t model. The test is \(H_0:\mu_d=0\) versus \(H_a:\mu_d\ne0\).
The mean difference is positive, so method A is higher on average in the sample by 0.400 units. The statistic agrees with the output: \(0.400/0.800=0.500\). Eight pairs produce \(df=8-1=7\).
A two-sided p-value of 0.6325 is not small: if the population mean difference were zero, a t statistic at least as far from zero as 0.500 would not be unusual under the model. At \(\alpha=0.05\), the result does not provide sufficient evidence to reject \(H_0\). This does not establish that the two methods have exactly equal population means.
The 95% interval includes zero and also includes both negative and positive differences. It is consistent with uncertainty about the direction of the population mean difference. The interval is in measurement units and concerns the population mean difference, not the range of method-to-method differences for individual objects.
Common Mistakes When Reading Output
- Ignoring the subtraction order. A negative value does not automatically mean “a decrease.” Check the output label and explain the sign using its exact order of subtraction.
- Confusing standard error with the spread of individual differences. The standard error describes the estimated sampling variability of the mean difference; \(s_d\) describes variation among the observed paired differences. They are related by \(SE=s_d/\sqrt{n}\), but they are not the same quantity.
- Reading a p-value without checking the alternative. A one-sided p-value applies to a specified direction. It cannot be interpreted as the p-value for a two-sided question, and the alternative should not be chosen after seeing the sign of the result.
- Treating a large p-value as proof of no difference. A large p-value says the data do not provide strong evidence against the null under the stated model. It does not prove that the population mean difference is zero.
- Interpreting a confidence interval as a range for most pairs. The interval estimates \(\mu_d\), the population mean difference. Individual paired differences can vary substantially beyond its endpoints.
- Using the number of measurements as the sample size. The paired t analysis uses the number of complete pairs. With \(n\) complete pairs, the ordinary test has \(df=n-1\).
Check Your Understanding
For each question, explain your reasoning rather than naming a software field alone.
- Software defines differences as follow-up minus baseline and reports a mean difference of \(-2.4\) units. What does the sign say about the sample averages?
- A paired analysis uses 18 complete pairs. What degrees of freedom should the ordinary paired t test report, and why?
- For a test of \(H_a:\mu_d>0\), software reports a one-sided p-value of 0.03. State what probability this p-value describes, assuming the null and model.
- A reported 95% confidence interval for a mean paired difference is \([-0.8,1.6]\) minutes. What parameter is being estimated, and does the interval include zero?
- Explain why a standard error of 0.5 units is not necessarily the typical distance between an individual pair’s two measurements.