Why the Pairing Must Be Preserved
Some measurements come in natural pairs: a value before and after an intervention for the same person, or results from two methods applied to the same specimen. The two measurements in each pair are related because they share the same person, specimen, or other observational unit. The question may concern the typical change within a unit, rather than a comparison between two unrelated collections of measurements.
A bootstrap interval for paired differences should reflect that design. First calculate one difference for each unit, using the same subtraction order throughout. Then resample the observed pairs as whole units—or, equivalently for an interval based only on those differences, resample the list of differences. Resampling the two sets of measurements independently breaks their observed matching and can produce differences that were never observed within any unit.
The target is a population summary of within-unit differences: for example, the population mean difference or the population median difference. Those are different parameters. A mean describes the average change; a median describes the midpoint of the population distribution of changes. The chosen statistic and interval should match the question.
The Paired Bootstrap Procedure
Suppose the observed data consist of \(n\) pairs \((X_i,Y_i)\). A bootstrap sample draws \(n\) of those observed pairs at random with replacement. Each selected pair is kept intact. For each bootstrap sample, calculate its \(n\) differences and then the statistic of interest, such as their mean or median. Repeating this process \(B\) times gives a bootstrap distribution of that statistic.
Because the difference is calculated within each selected pair, resampling intact pairs and resampling the observed differences are equivalent for these statistics. For example, drawing pair numbers \(2,2,5,1\) means using the second pair twice, the fifth pair once, and the first pair once, and calculating a difference within each selected pair. Equivalently, draw \(D_2,D_2,D_5,D_1\) from the list of observed differences.
A percentile interval uses the lower and upper quantiles of the bootstrap statistics. For a nominal 95% interval, these are the 2.5th and 97.5th percentiles. This is an approximate confidence interval: its repeated-sampling coverage is not guaranteed to be exactly 95%, and its quality depends on whether the data and bootstrap approximation are suitable.
Here \(Q_{0.025}^{*}\) and \(Q_{0.975}^{*}\) are the 2.5th and 97.5th percentiles of the \(B\) bootstrap values of the chosen statistic. The asterisk marks a bootstrap quantity. Other bootstrap interval methods can also be applied to paired statistics, but the essential design rule remains: each resample must preserve the pairing.
Worked Example: A Percentile Interval for the Mean Change
In a hypothetical study, six people have a measurement recorded before and after a program. Define change as after minus before, so a positive difference means an increase. The readings are in millimetres of mercury (mmHg).
| Person | Before | After | Difference: after − before |
|---|---|---|---|
| 1 | 120 | 121 | 1 |
| 2 | 128 | 130 | 2 |
| 3 | 135 | 137 | 2 |
| 4 | 140 | 143 | 3 |
| 5 | 150 | 154 | 4 |
| 6 | 160 | 166 | 6 |
The observed differences are \(1,2,2,3,4,6\) mmHg. Their sample mean is
For a bootstrap draw, suppose the selected person numbers are \(1,3,3,5,2,6\). Keeping each selected pair intact gives differences \(1,2,2,4,2,6\), with bootstrap mean
Repeating the draw 10,000 times in a hypothetical bootstrap run produces a 95% percentile interval of approximately 1.67 to 4.50 mmHg. This interval estimates the population mean of the after-minus-before differences. In context, it is compatible with a mean increase between about 1.67 and 4.50 mmHg. It does not establish that the program caused the change; that conclusion depends on the study design and other possible explanations.
The observations are paired because each before reading belongs to the same person as its after reading. For the bootstrap to represent sampling uncertainty, the sampled people should also reasonably represent independent units from the population of interest. If the people were selected in a strongly nonrepresentative way, resampling cannot repair that limitation.
Intervals for a Median Difference
The procedure is the same when the target is the population median difference: resample the observed differences, calculate the median in each bootstrap sample, and use the relevant bootstrap percentiles. The median may be useful when the question concerns a typical within-unit change and the difference distribution is skewed or contains extreme values. It is not interchangeable with the mean; state which population summary the interval concerns.
Worked Example: A Percentile Interval for the Median Change
Suppose seven hypothetical patients have paired symptom scores recorded before and after an intervention. Define improvement as before minus after, so positive values mean lower symptom scores after the intervention. The observed improvements, in score points, are \(-2,0,1,2,3,5,8\). The ordered values show that the sample median improvement is 2 points.
One bootstrap sample might select the observed differences \(-2,1,2,2,5,8,0\). Once ordered, these are \(-2,0,1,2,2,5,8\), so the bootstrap median is 2 points. A different draw can repeat some observed differences and omit others; its median may therefore differ.
For this small example, consider the exact bootstrap distribution formed by all \(7^7\) equally likely ordered draws of seven values with replacement. Its 2.5th percentile is 0 points and its 97.5th percentile is 5 points, giving a 95% percentile interval of \([0,5]\) points. This interval concerns the population median improvement, not the population mean improvement. With only seven observed pairs, it should be treated cautiously: the interval has limited resolution, and the empirical distribution contains only seven distinct observed values.
Why Independent Resampling Changes the Question
The reason intact resampling matters can be seen by writing a paired difference as \(D=Y-X\). Its variance is
The covariance term records how the two measurements vary together within pairs. If the measurements tend to rise and fall together, the covariance is positive and the variation in their differences can be much smaller than the variation in either measurement alone. Resampling the \(X\) values separately from the \(Y\) values combines measurements from different units. It removes the observed pairing and does not preserve the within-pair covariance.
This is not a minor change in computational style. It creates artificial pairings and estimates uncertainty for a different arrangement of the data. For a question about within-unit change, the observed pair—not either measurement in isolation—is the sampling unit.
Worked Example: Keeping the Same Person Together
Consider five hypothetical paired readings, in arbitrary units: \(X=(100,110,120,130,140)\) and \(Y=(102,113,121,134,145)\). Define \(D=Y-X\). The observed differences are \(2,3,1,4,5\), with mean
Suppose one bootstrap draw selects units \(1,3,3,5,2\). Resampling the intact pairs gives differences \(2,1,1,5,3\), with mean \(12/5=2.4\). These are differences that retain each selected unit’s observed relationship between \(X\) and \(Y\).
Now imagine instead that the \(X\) values are drawn using unit numbers \(1,3,3,5,2\), while the \(Y\) values are independently drawn using unit numbers \(2,1,5,3,3\). Subtracting the separately drawn \(X\) values from the \(Y\) values gives \(13,-18,25,-19,11\), with mean \(12/5=2.4\). The matching is now artificial: for example, the first difference combines \(Y_2=113\) with \(X_1=100\), although those readings did not come from the same unit. A matching mean for this one draw does not make the procedures equivalent; their bootstrap distributions can differ because independent resampling discards the within-pair relationship.
The correct resampling scheme follows the design and the target. Here the target is a summary of within-unit differences, so whole pairs—or the already calculated differences—must be resampled together.
Conditions and Common Mistakes
A paired bootstrap interval is an approximation, not a guarantee that every dataset will give a reliable interval. Before interpreting one, check the design and data:
- There is a genuine match. Each \(X_i\) and \(Y_i\) belong to the same unit or matched unit, and the scientific question concerns their within-pair difference.
- The pairs are appropriate sampling units. The observed pairs should be independent of one another, or their dependence must be handled by a method suited to the design. For example, repeated observations clustered within the same person are not automatically independent pairs.
- The sample represents the target population. The bootstrap resamples the observed data; it cannot correct selection bias, measurement problems, or an unrepresentative sample.
- The difference data support the approximation. With very small samples, strong skewness, or influential extreme differences, the bootstrap distribution and interval may be unstable. More bootstrap draws reduce simulation noise, but do not fix limited or poor data.
- The subtraction order is consistent. Reversing the order changes the sign of every difference and reverses the interval endpoints. It does not change the underlying pairing.
A frequent mistake is to say, “There are two measurements, so I should bootstrap each column separately.” That reasoning treats the columns as unrelated groups and ignores the study’s matched design. The correction is to identify the observational unit first, calculate one difference per unit, and resample those differences (or resample intact pairs). A second mistake is to interpret a 95% interval as a 95% probability that the fixed population parameter lies in this particular interval. The correct interpretation is that, under the method’s assumptions, intervals produced by repeated sampling and the same procedure would capture the parameter about 95% of the time.
Check Your Understanding
Answer each question using the paired-bootstrap logic from this tutorial.
- For measurements before and after a treatment, what does defining \(D_i=\text{after}_i-\text{before}_i\) make a positive difference mean?
- Why does resampling the observed differences give the same bootstrap statistics as resampling intact pairs when the statistic depends only on those differences?
- A researcher wants an interval for the population median within-person change. What statistic should be calculated for each bootstrap resample?
- Explain how independently resampling two measurement columns can alter the variability of their differences.
- What does a 95% bootstrap confidence interval mean under the method’s assumptions, and what does it not mean?