Why One Missing Measurement Matters
A paired design gets information from the relationship between two measurements belonging together: for example, a before-and-after measurement on one person, or measurements on two participants who were matched. The usual paired analysis first calculates a difference within each pair. If either measurement is missing, that pair’s difference cannot be calculated from the observed data.
This does not mean the whole study is unusable. It does mean that the analyst must be clear about which observations support the result. A complete-pair analysis uses only pairs with both measurements observed. Other analyses may retain individual measurements from incomplete pairs, but they can target a different quantity and need assumptions appropriate to the design and the missingness. Neither choice can recover an unobserved value by itself.
What a Complete-Pair Analysis Uses
Suppose pair \(i\) has measurements \(X_i\) and \(Y_i\), and the difference is defined as \(D_i=X_i-Y_i\). The sign convention matters: positive differences mean the \(X\) measurement exceeds the \(Y\) measurement. A missing \(X_i\) or \(Y_i\) means \(D_i\) is unknown, not zero.
If \(m\) pairs are complete, the sample mean difference is the average of their \(m\) observed differences. The paired standard error describes the sampling variability of that mean under the paired model; it uses the variation among the observed pair differences, not the variation among all individual measurements treated as unrelated.
Here \(R_i=1\) means pair \(i\) is complete, and \(s_D\) is the sample standard deviation of the \(m\) observed differences. Inference using a paired \(t\) procedure assumes that the pair differences are independent across pairs and that the sampling or assignment design supports the intended population comparison. For a small sample, the distribution of the differences should also be reasonably free of severe skewness or outliers; with larger samples, the procedure is more tolerant of non-normality.
Worked Example: Calculating Differences From the Complete Pairs
In a hypothetical matched study, a quantitative score is recorded under conditions \(X\) and \(Y\) for six pairs. The difference is \(X-Y\), so a positive value favors a higher score under \(X\). Pair F is missing its \(Y\) score.
| Pair | Score under \(X\) | Score under \(Y\) | Observed difference \(X-Y\) |
|---|---|---|---|
| A | 72 | 70 | 2 |
| B | 68 | 69 | -1 |
| C | 75 | 72 | 3 |
| D | 71 | 71 | 0 |
| E | 66 | 62 | 4 |
| F | 73 | Missing | Not calculable |
There are five complete pairs, so \(m=5\). Pair F does not contribute a difference: its difference is not zero, but unknown. For the five observed differences,
The sample standard deviation of these differences is \(s_D=\sqrt{17.2/4}=2.074\) score points, so the standard error is
These calculations describe the five complete pairs. If the pairs were independent and the difference distribution were suitable for the intended inference, a paired procedure could use them. With only five differences, the data alone provide little reassurance about distribution shape; the study design and a plot of the differences would also matter.
When Complete Pairs Represent the Intended Target
The mean difference among complete pairs is always a descriptive summary of those pairs. To use it as an estimate of a broader population’s mean paired difference requires more: the complete pairs must be sufficiently representative of that target population. A useful condition is that completion is unrelated to the pair difference, or at least does not change its mean. If pairs with certain differences are more likely to be incomplete, the complete-pair mean may systematically miss those differences.
This is the central risk of selective missingness: the chance that a pair is complete depends on information related to the outcome difference. For example, follow-up measurements may be harder to obtain from participants whose health worsens, or a device may fail more often at extreme readings. The observed complete pairs can then differ in a meaningful way from all planned pairs.
The connection can be stated precisely. Let \(D\) be the difference for a pair in the target population, and let \(R=1\) indicate that the pair is complete. If \(P(R=1)>0\), then
To see why, \(E[DR]=P(R=1)E[D\mid R=1]\) and \(E[R]=P(R=1)\). Substituting these into \(\operatorname{Cov}(D,R)=E[DR]-E[D]E[R]\) gives the equation. Thus, if completion is associated with the difference, the mean difference among complete pairs can differ from the overall mean. When completion is unrelated to \(D\), the covariance is zero and the two means agree in expectation. This condition is a justification, not something guaranteed merely because the analysis uses only complete pairs.
Worked Example: Selective Missingness Can Reverse a Result
Suppose eight pairs were planned in a hypothetical before-and-after study, with each difference defined as after minus before. Four complete pairs have observed differences of \(-4\), \(-3\), \(-2\), and \(-1\) units. The other four pairs are missing their after measurement because participants with worsening outcomes were less likely to return.
The complete-pair estimate is
Among the observed complete pairs, the average change is a decrease of 2.50 units. But this does not establish that the average change across all eight planned pairs was a decrease. To illustrate the risk, suppose the four unobserved differences had been \(2\), \(3\), \(4\), and \(5\) units. Those values are hypothetical and cannot be inferred from the observed data; they show one possible pattern consistent with worse outcomes being more likely to go missing. The mean across all eight would then be
That hypothetical overall mean is an increase, not a decrease. The actual missing differences are unknown, so the example does not tell us which mean is correct. It shows why a complete-case result can be misleading when completion is related to outcome. The reason for missingness should be investigated and reported; the observed differences alone cannot diagnose every form of selective missingness.
Retaining Measurements From Incomplete Pairs
A different strategy is to retain all observed individual measurements, including the one observed value from an incomplete pair. For example, analysts might compare the mean of all observed \(X\) measurements with the mean of all observed \(Y\) measurements. This is not the same calculation as averaging within-pair differences among complete pairs. It may be useful for a question about a marginal contrast between the two conditions, but its interpretation and validity depend on the design and on how missingness occurred.
In particular, matching creates a relationship between observations in the same pair. An analysis that retains individual measurements must not automatically treat every measurement as independent. It must use a method appropriate to the dependence and to the way pairs were sampled, formed, or assigned. Nor does retaining more numbers automatically remove bias: if missingness changes which people contribute measurements to one condition, that condition’s observed mean may not represent its intended population mean.
Worked Example: The Available-Measurement Contrast Is Not the Paired Mean
Consider five hypothetical matched pairs, with quantitative scores under conditions \(X\) and \(Y\). Two pairs are incomplete. Their observed measurements are shown below.
| Pair | \(X\) score | \(Y\) score | Complete difference \(X-Y\) |
|---|---|---|---|
| 1 | 12 | 10 | 2 |
| 2 | 15 | 14 | 1 |
| 3 | 11 | 9 | 2 |
| 4 | Missing | 2 | Incomplete |
| 5 | 25 | Missing | Incomplete |
The complete-pair estimate uses only the first three differences:
If we instead calculate a contrast using every observed score, the \(X\) mean is based on four measurements and the \(Y\) mean is based on four measurements:
The observed marginal contrast is \(15.75-8.75=7.00\) score points, quite different from \(1.667\). The difference arises because the two calculations use different information and summarize different features of these incomplete data. In a real analysis, one would need to ask whether the observed measurements represent the intended conditions and use a method that accounts for the matched design. Simply calling the \(7.00\)-point contrast a paired estimate, or treating its measurements as independent without justification, would be incorrect.
Practical Decisions and Reporting
There is no universal rule that every study with an incomplete pair must use one analysis. The choice should follow the research question, design, and plausible missingness process. A complete-pair analysis is direct when the target is a mean within-pair difference and the complete pairs reasonably represent the intended population. An analysis retaining unpaired observations may be appropriate when the target is a marginal contrast and the method correctly reflects the design and dependence. The assumptions behind either choice should be made explicit.
Do not fill a missing measurement with zero, the other member’s measurement, or a convenient average merely to create a difference. Such substitutions impose values not observed and can alter both the estimate and its uncertainty. More advanced methods may address incomplete data under additional assumptions, but none can make those assumptions unnecessary.
At minimum, report the number of pairs planned, the number complete, which measurements were missing, and how the primary analysis handled incomplete pairs. If possible, describe when and why measurements were missed and compare relevant observed characteristics of complete and incomplete pairs. These details help readers assess whether the complete pairs plausibly represent the target population; they do not prove that missingness is harmless.
Common Mistakes
- Using an incomplete pair’s observed value as a difference. A difference requires both measurements. Keep the observed value for a suitable analysis, but do not invent its partner measurement.
- Assuming complete cases are representative because they are the only usable pairs. Their mean describes the complete pairs; generalizing it requires a defensible assumption about how pairs became complete.
- Treating a larger number of measurements as automatically better. Retaining individual measurements can change the target and must still account for dependence within matched pairs.
- Claiming that missingness caused bias based only on the fact that data are missing. Missingness creates a risk, not proof of bias. The concern is strongest when completion is related to outcomes or characteristics that affect them.
Check Your Understanding
For each question, distinguish what is directly observed from what requires an assumption.
- A pair has an observed \(X\) value of 18 and a missing \(Y\) value. What can and cannot be calculated for this pair?
- In a study of 40 planned pairs, 31 are complete. What group of pairs does the complete-pair mean directly describe?
- Why might a mean contrast based on every observed \(X\) and \(Y\) measurement differ from the mean of complete-pair differences?
- Suppose participants with unusually large changes are less likely to return for the second measurement. Why could the complete-pair mean be unrepresentative?
- What information about incomplete pairs should be reported alongside a complete-pair analysis?