Why Compare Within Pairs?
Suppose a researcher wants to compare two study methods using students’ quiz scores. Students differ in prior knowledge, attention, and familiarity with the material. If one group happens to contain more students with strong prior knowledge, an observed score difference could reflect that imbalance as well as the study methods. A matched-pairs design addresses this problem by making comparisons within pairs of similar units, or within the same unit under two conditions.
The purpose is not simply to collect data in twos. The design makes the comparison between treatments the main focus: for each pair, one outcome is compared with the other. When members of a pair are similar in ways that affect the response, their shared characteristics can contribute less variation to that comparison.
Two Ways to Form the Pairs
In the first approach, the researcher forms pairs of distinct experimental units that are alike on characteristics expected to affect the response. For example, students might be paired by similar baseline quiz scores. Within each pair, one student is randomly assigned to Treatment A and the other to Treatment B. The two students are not identical, but the pairing aims to make their outcomes comparable.
In the second approach, the same unit receives both treatments, with an outcome recorded under each. For example, each participant might complete two comparable tasks, one using each study method. This design controls for stable differences between participants because each person serves as their own comparison. However, treatment order must be considered. If practice, fatigue, or lingering effects of the first treatment influence the second outcome, the comparison may be distorted. Randomizing treatment order can help address order effects; when a treatment’s effect carries over, order randomization alone may not be sufficient.
| Pairing approach | What is paired? | Important design consideration |
|---|---|---|
| Two matched units | Two distinct units with similar relevant characteristics | Randomly assign one unit in each pair to each treatment |
| Same unit under both treatments | Two measurements from one unit | Consider order, practice, fatigue, and carryover effects |
| Before-and-after measurements | A unit’s outcome before and after an intervention | Change over time may also explain the difference |
The third row needs special care. Measuring the same participants before and after an intervention creates paired observations, but it does not by itself make the study a randomized matched-pairs experiment. If everyone is measured before and then after, treatment condition is tied to time. The observed change may be due to the intervention, but it could also reflect other changes over time. Pairing controls for stable differences between participants; it does not automatically control for time-related influences.
Random Assignment Within Pairs
For a two-unit pair, a common procedure is to use chance to assign one unit to Treatment A and the other to Treatment B. Each pair receives both treatments, but which member receives which treatment is determined randomly. Randomization helps prevent a systematic assignment pattern, such as always giving the treatment expected to work better to the stronger member of each pair.
A good pair is similar on characteristics related to the response, not merely similar in an incidental way. If baseline skill strongly predicts quiz score, pairing students with similar baseline skill may be useful. Pairing them by an unrelated characteristic may not reduce much variation. The outcome and the practical context should guide the choice of matching characteristics.
Identify characteristics likely to affect the outcome, then form pairs that are similar on those characteristics, or plan to measure both treatments on each unit.
For each pair of distinct units, use chance to assign one member to A and the other to B. For repeated measurements, consider randomizing the order of treatments.
For every pair, record both outcomes and define the difference in a consistent direction, such as outcome under B minus outcome under A.
Summarize how the paired differences vary and what their direction means. Consider whether the design supports a causal comparison and what limitations remain.
Paired Differences and Their Variability
Let \(Y_{iA}\) and \(Y_{iB}\) denote the outcomes under Treatments A and B for pair \(i\). Define the paired difference as \(D_i=Y_{iB}-Y_{iA}\). The order matters: with this definition, a positive \(D_i\) means the outcome was higher under B than under A. Reversing the subtraction reverses the signs, but not the basic comparison.
For \(n\) pairs, the average paired difference is \(\bar{D}=\frac{1}{n}\sum_{i=1}^{n}D_i\). This summarizes the observed within-pair treatment differences. A matched-pairs analysis focuses on these differences rather than treating all outcomes under A and all outcomes under B as unrelated groups.
The first formula explains the potential advantage of pairing. When the outcomes within a pair tend to move together, \(\rho\) is positive, and the variance of their difference can be smaller than it would be if the outcomes were unrelated. Pairing is not guaranteed to reduce variability: if pairs are poorly chosen or the within-pair association is weak, the gain may be small.
Why the Variance Formula Holds
For any two variables \(Y_{iA}\) and \(Y_{iB}\), the variance of their difference is the sum of their variances minus twice their covariance. If each has variance \(\sigma^2\), and their correlation is \(\rho\), their covariance is \(\rho\sigma^2\). Substitution gives the within-pair result:
To obtain the variance of the average difference, further conditions are needed. If the \(D_i\) are uncorrelated across pairs and all have variance \(\operatorname{Var}(D_i)\), then
The middle equality relies on the across-pair differences being uncorrelated; it does not follow just from the outcomes within each pair having correlation \(\rho\). In particular, dependent pairs can make the average more or less variable than this formula indicates. The usual design rationale assumes that different pairs are formed and treated independently, so their differences do not share a source of dependence.
Worked Example: Randomizing Treatment Within Matched Student Pairs
A researcher wants to compare two short study methods using quiz scores. Twelve students are grouped into six pairs with similar baseline quiz scores. Within each pair, a coin toss determines which student uses Method A and which uses Method B. Each student studies the same material for the same length of time, and then takes a comparable quiz. Identify the design and explain what a paired difference would mean.
The experimental units are the students, and the treatments are the two study methods. The six pairs are formed using baseline scores, a characteristic expected to relate to quiz performance. Random assignment within each pair prevents the researcher from systematically assigning one method to the student with the higher baseline score.
For each pair \(i\), define \(D_i\) as the Method B student’s quiz score minus the Method A student’s quiz score. If one pair’s scores are 82 under B and 78 under A, its difference is \(D_i=82-78=4\) points. This pair’s observed score was 4 points higher under B. The average of the six differences would summarize the average within-pair score comparison, measured in quiz points.
The design supports a causal comparison of the assigned methods if assignment is carried out as planned, outcomes are measured comparably, and one student’s treatment does not affect another student’s outcome. The pairing is useful if students with similar baseline scores tend to have similar quiz outcomes. It does not ensure that every pair has a small difference or that Method B is better.
Worked Example: Summarizing Four Paired Differences
In a hypothetical experiment, four matched pairs of units are assigned one each to Treatment A and Treatment B. The measured outcomes, in units of a response score, are shown below. The analysis defines each difference as B minus A.
| Pair | Outcome under A | Outcome under B | \(D_i=B-A\) |
|---|---|---|---|
| 1 | 10 | 12 | 2 |
| 2 | 8 | 11 | 3 |
| 3 | 13 | 14 | 1 |
| 4 | 9 | 13 | 4 |
The four paired differences are \(2,3,1,4\). Their average is
Thus, in these four observed pairs, the outcome under B was higher by an average of 2.5 response-score units. To describe the spread of the differences, the sample standard deviation is calculated from deviations around \(\bar{D}=2.5\):
This standard deviation describes the variation among the four paired differences, in response-score units. It is not the standard deviation of all eight individual outcomes. The example summarizes observed data; by itself, it does not establish how precisely the average difference estimates an effect in a broader population.
Before-and-After Measurements Require Caution
Worked Example: Interpreting a Before-and-After Comparison
A hypothetical clinic records a symptom score for each of 20 participants, offers all participants a new program, and records the score again four weeks later. For one participant, the score falls from 18 before the program to 13 afterward. With change defined as after minus before, that participant’s paired difference is \(13-18=-5\) score points. If the average change across all participants is also negative, the scores decreased on average during the period.
Each participant provides a pair of measurements, so the differences account for stable participant characteristics when summarizing change. But this is not, on its own, a randomized comparison of the program with a control treatment. The scores might have changed because of the program, but they could also have changed because of natural recovery, other events during the four weeks, or changes in measurement. The paired structure describes how observations are connected; it does not by itself establish the cause of the change.
When Matching Helps—and When It Does Not
A matched-pairs design is most useful when paired outcomes are expected to be positively associated for meaningful reasons. For example, two people with similar baseline ability may have similar outcomes regardless of which treatment they receive. Comparing their outcomes can then remove some of the variation due to ability. Likewise, when the same participant receives both treatments under suitable conditions, stable differences between participants do not contribute to the within-person contrast.
Matching also has practical limits. It may be difficult to find suitable partners, and pair formation may not make outcomes strongly similar. The choice of matching criteria should be made before examining treatment outcomes; otherwise, the researcher may create pairs based on patterns that arose by chance. In a repeated-treatment design, the same unit’s second response can be influenced by the first treatment. In either form, the experiment still requires careful measurement, a clear assignment procedure, and attention to interference or other design problems.
Common Mistakes
- Assuming that any two measurements make a matched-pairs experiment. Two related measurements are paired data, but a before-and-after study with no randomized treatment comparison may not support a causal conclusion about the intervention.
- Ignoring the direction of subtraction. If differences are defined as B minus A, a positive value means B produced the higher outcome. State the order before interpreting signs.
- Assuming matching always reduces variability. Pairing helps when within-pair outcomes are positively related. Weak or unhelpful matching may yield little benefit.
- Using a within-pair correlation to justify independence across pairs. Correlation between A and B within one pair says nothing by itself about dependence between different pairs. The variance formula for \(\bar{D}\) additionally requires uncorrelated paired differences with equal variance.
- Forgetting order effects when each unit receives both treatments. Practice, fatigue, or carryover may affect the second measurement, so treatment order and the conditions for comparing outcomes must be considered.
Check Your Understanding
Try these questions without looking back.
- A researcher pairs participants with similar baseline running times and randomly assigns one person in each pair to each of two training plans. What is the experimental unit, and where does random assignment occur?
- For a pair, the outcome is 24 under Treatment A and 20 under Treatment B. What is the paired difference if it is defined as B minus A, and what does its sign mean?
- Why can a before-and-after study control for stable differences between participants but still fail to isolate the causal effect of an intervention?
- In the variance formula for the mean paired difference, what additional condition about differences from different pairs is needed?
- Give one reason a repeated-measures comparison might be distorted if every participant receives Treatment A before Treatment B.