Why Paired Inference Needs Diagnostics
A paired analysis turns each pair into one difference and applies one-sample t inference to those differences. That makes the pairing useful, but it also makes the assumptions specific: the relevant observations are the differences, and those differences must provide independent information about the population mean difference. A plot can help assess their shape, but it cannot establish that the pairs were independent in how they were collected.
For example, when the same person is measured before and after an intervention, the two measurements within that person are expected to be related. This is not a violation of the paired t model: the two measurements are combined into one difference. The important independence question is whether one person’s difference is independent of another person’s difference.
The target parameter is the population mean difference, denoted \(\mu_d\). The paired t statistic is the one-sample statistic calculated from the differences:
Here, \(\bar{d}\) is the sample mean of the differences and \(s_d\) is their sample standard deviation. Diagnostics are not a separate calculation that certifies the assumptions. They combine knowledge of the study design with an inspection of the observed differences to judge whether this model is credible.
Condition 1: Independent Pairs
Independence is primarily a question about how the data were obtained. If pairs are selected independently from a population, or if pairs are independently assigned in an experiment, it may be reasonable to treat their differences as independent. The exact justification depends on the design. A random sample of people, for instance, does not automatically make observations independent if the people were sampled in strongly connected groups and outcomes within each group tend to move together.
Dependence within a pair is not only allowed but often the reason for pairing. In a before-and-after study, two measurements on the same person are linked; in a matched study, the members of a pair may be similar. The paired analysis accounts for that connection by using one difference per pair. It does not account for dependence between pairs. If two pairs share a participant, belong to the same small household, or come from the same group in a way that makes their differences correlated, the usual independent-differences model may fail.
The unit that contributes one difference must also be clear. If a study measures several outcomes or several time points for each person, counting all of those measurements as independent pairs can exaggerate the amount of independent information. The independence assessment should follow the design and the actual units, not simply the number of rows in a file.
Worked Example: Checking Independence in Matched Pairs
Suppose a hypothetical study matches four patients into pairs and records each patient’s change in a symptom score after a program. Within each pair, the difference is calculated as patient A’s change minus patient B’s change. The four observed differences, in score points, are \(2,-1,3,0\).
The sample mean difference is:
This calculation is meaningful as a summary of the four pair differences, but it does not establish independence. If each patient belongs to only one pair and the four pairs were formed and recruited separately, treating the pair differences as independent may be reasonable. If, instead, the same patient appears in two pairs, or all four pairs come from a tightly connected group with shared influences, the differences may be dependent. The design information—not the value \(1\), nor a graph of the four numbers—must address that concern.
Independence cannot be verified by looking at a histogram or a normal probability plot. Those plots show the values and shape of the observed differences; they do not reveal whether observations were linked by the sampling or experimental design. Conversely, a sound sampling design does not guarantee that the differences are approximately normal. These are distinct checks.
Condition 2: The Distribution of Differences
For exact small-sample t inference, the differences must be independent and identically distributed normal observations. In notation, the model is \(d_i\sim N(\mu_d,\sigma_d^2)\) for \(i=1,\ldots,n\), with the \(d_i\) independent. The second parameter in \(N(\mu_d,\sigma_d^2)\) is the variance. If the differences are only approximately normal, t inference is an approximation, not exact inference.
In practice, a histogram, dotplot, or boxplot of the differences can show strong asymmetry, gaps, or observations far from the rest. A normal probability plot can also help assess whether the data depart substantially from a normal shape. The focus is on the differences, not on each original measurement considered separately: a paired t procedure is based on the difference distribution.
A severe outlier deserves attention because \(\bar{d}\) and \(s_d\) can both be sensitive to an extreme value. An unusual difference can reflect a data-entry or pairing error, a genuine but rare outcome, or a feature of the population. It should be investigated and reported as appropriate, not deleted merely because removing it makes the data look more normal.
Small samples provide limited evidence about distribution shape: a plot with few points may not reveal departures from normality. With a larger sample, the t procedure can often tolerate moderate nonnormality, especially when the difference distribution is not strongly skewed and has no very influential outliers. This is a practical robustness principle, not a rule that a particular sample size makes any distribution safe. Strong skewness or extreme observations remain concerns, and dependence is not repaired by increasing \(n\).
Worked Example: An Influential Difference
Consider these hypothetical paired differences in minutes for eight pairs, using treatment minus comparison: \(1,2,2,3,3,4,4,13\). The value \(13\) is much larger than the other differences, so we calculate the center and spread to see why it warrants investigation.
The mean is:
The sum of squared deviations from the mean is:
Thus the sample standard deviation is:
The difference \(13\) is a plausible influential observation: it is far from the other seven values and contributes \(81\) of the \(100\) squared-deviation units. The calculations do not prove that it is an error, and they do not decide whether an inference is acceptable. The next steps are to verify the pair and the recorded values, inspect the study context, and describe the impact of the observation when interpreting the diagnostic.
Why the Normality Condition Gives a t Distribution
The exact t result depends on the normal model, not merely on a symmetric-looking plot. Here is the reason. If \(d_1,\ldots,d_n\) are independent observations from \(N(\mu_d,\sigma_d^2)\), then the standardized sample mean
has a standard normal distribution. Under the same model, the scaled sample variance
has a chi-square distribution with \(n-1\) degrees of freedom and is independent of \(Z\). The t distribution with \(n-1\) degrees of freedom is defined by the ratio \(Z/\sqrt{U/(n-1)}\). Substituting the expressions for \(Z\) and \(U\) gives:
Therefore, the paired t statistic has exactly a \(t\) distribution with \(n-1\) degrees of freedom under independent, identically distributed normal differences. If the differences are only approximately normal, this derivation does not make the result exact; using the t distribution is then an approximation whose quality depends on how severe the departures are and on the sample size.
Worked Example: A Small, Fairly Balanced Difference Sample
A hypothetical study records ten paired changes, in points: \(-3,-2,-2,-1,-1,0,0,1,1,2\). A dotplot would show values extending on both sides of zero without one point separated far from the others. We can check the arithmetic summaries alongside that visual inspection.
The mean difference is:
The sum of squared deviations can be calculated from \(\sum d_i^2-n\bar{d}^{\,2}\). Here \(\sum d_i^2=25\) and \(n\bar{d}^{\,2}=10(0.25)=2.5\), so:
The displayed values show no conspicuous isolated outlier and are reasonably balanced around the center, so there is no obvious severe skewness in this small sample. Still, ten values cannot establish that the population of differences is normal. If t inference is used, the distributional justification is approximate unless the normal model is known to hold; the diagnostic only says that these data do not show a major warning sign.
A Practical Diagnostic Sequence
A disciplined check separates design questions from distribution questions. The following sequence makes the reasoning explicit before relying on paired t inference.
Check that each row represents one legitimate pair, that subtraction follows the stated order, and that each pair contributes one difference.
Ask whether different pairs were sampled or assigned independently and whether people, households, sites, or other units are shared or clustered in a way that could link their differences.
Use a dotplot, histogram, boxplot, or normal probability plot to look for strong skewness, separated clusters, and unusual extremes.
Check records and context, but do not remove a valid observation solely to improve the plot. Consider whether conclusions would be sensitive to it.
Distinguish a normal model that supports exact t inference from an approximately normal sample that supports only an approximate procedure.
Common Mistake: Treating a Plot as Proof
A mistaken argument is: “The histogram of differences looks normal, so the paired observations are independent and the t result is exact.” The plot cannot show independence; that comes from the sampling or experimental structure. Also, a plot from a finite sample cannot establish exact population normality. The correction is to assess independence from the design, use plots to identify distributional concerns, and describe exactness accurately: independent, identically distributed normal differences give exact small-sample t inference, while approximate normality supports only an approximation.
Another common error is checking the two sets of measurements separately and ignoring the differences. Paired t inference concerns the distribution of the within-pair differences. The original measurements may each look skewed while their differences are fairly well behaved, or the reverse may occur. The relevant diagnostic is the one that matches the model.
Check Your Understanding
Answer each question by referring to the paired differences and the study design.
- In a before-and-after study, measurements on the same person are related. Why does that relationship not by itself violate the independence condition for paired t inference?
- A study has one difference per household, but several households share the same small residential facility. What information would you need to assess whether the differences can be treated as independent?
- Why should a diagnostic plot for paired t inference show the differences rather than only the before and after measurements separately?
- A small sample of differences has a strong right skew and one very large value. What should be checked before deciding how to proceed, and why is automatic deletion inappropriate?
- State the distributional assumption for exact small-sample t inference. How should the conclusion be worded if the differences are only approximately normal?