Why Analyze Matched Outcomes as Pairs?
A 1:1 propensity-score match creates pairs, each containing one treated person and one comparison person. For a continuous outcome, the central analysis is to calculate the treated-minus-control difference within each pair, then summarize those differences. This respects the design: each comparison is made between two people selected to be similar on measured pretreatment characteristics, rather than treating the two outcome groups as unrelated.
The paired calculation does not, by itself, make treatment assignment random or guarantee a causal conclusion. Matching addresses measured covariate differences to the extent that the match succeeds. A causal interpretation also requires assumptions about confounding, treatment definition, and the matched population. The estimate and its uncertainty should therefore be reported separately from the assumptions that give them a causal interpretation.
The units of \(D_k\) and \(\bar{D}\) are the units of the outcome. If the outcome is a symptom score, the estimate is in score points; if it is length of stay, it is in days. The pair differences should be defined in the same direction throughout. Switching to control-minus-treated reverses the sign of the estimate and all interval endpoints.
Estimand and Assumptions
In potential-outcome notation, let \(Y_i(1)\) and \(Y_i(0)\) denote person \(i\)'s outcomes under treatment and control, and let \(A_i\) indicate observed treatment status. The average treatment effect among treated people is \(E[Y(1)-Y(0)\mid A=1]\). When matching retains all treated people, the average paired difference is intended to estimate this effect for those treated people represented in the matched sample. If treated people are discarded, for example because of limited overlap, the target is instead the treated population retained by the procedure. It should not be described as the effect for all treated people without justification.
A causal interpretation relies on conditions beyond the paired calculation. Consistency requires that the observed outcome equal the potential outcome under the treatment actually received, with treatment and outcome defined meaningfully. No interference requires that one person's treatment not alter another person's outcome. Conditional exchangeability requires that, given measured pretreatment covariates \(X\), treatment assignment be independent of the potential outcomes. Positivity requires a nonzero chance of either treatment level for covariate patterns in the target population. In practice, the matching procedure must also achieve adequate balance and avoid relying on comparisons outside the region of covariate overlap. These conditions cannot be established merely by observing a small paired standard error.
For the usual paired t interval and test, the data are treated as \(m\) independent pair differences with a common mean and variance. Exact small-sample t inference requires the pair differences to be independently normally distributed. Approximate normality may support an approximate t procedure, particularly as the number of pairs grows, but it does not make the procedure exact. The separate outcome distributions for treated and comparison people need not each be normal for a paired analysis; it is the distribution of the differences that matters.
Paired Estimate, Standard Error, and Interval
Let \(s_D\) be the sample standard deviation of the \(m\) pair differences. The estimated standard error of their mean is \(SE(\bar{D})=s_D/\sqrt{m}\). Under the independent normal-differences model, the statistic for testing \(H_0:\mu_D=0\), where \(\mu_D\) is the population mean difference in the modeled matched-pair population, follows a t distribution with \(df=m-1\). The same distribution gives a two-sided confidence interval.
Here \(\mu_{D,0}\) is the hypothesized mean difference, usually zero when testing for no average difference, and \(t_{1-\alpha/2,\;m-1}\) is the relevant t critical value. A 95% interval describes values of the mean difference compatible with the data and model at that confidence level. It is not a statement that there is a 95% probability the fixed population parameter lies in this particular interval.
Worked Example: A Paired Estimate and 95% Interval
Worked Example: A Paired Estimate and 95% Interval
Suppose a hypothetical matched analysis compares a treatment with usual care on a continuous symptom score, measured in points. The following six pairs are retained. The numbers are illustrative, not results from a real study.
| Pair | Treated score | Comparison score | \(D_k\): treated minus comparison |
|---|---|---|---|
| 1 | 12 | 10 | 2 |
| 2 | 16 | 12 | 4 |
| 3 | 10 | 9 | 1 |
| 4 | 17 | 14 | 3 |
| 5 | 11 | 11 | 0 |
| 6 | 15 | 13 | 2 |
The differences are \(2,4,1,3,0,2\) points, so \(\bar{D}=(2+4+1+3+0+2)/6=2.00\) points. Their deviations from 2 are \(0,2,-1,1,-2,0\), and the sum of squared deviations is \(10\). Thus \(s_D=\sqrt{10/(6-1)}=\sqrt{2}=1.414\) points, and \(SE(\bar{D})=1.414/\sqrt{6}=0.5774\) points.
There are \(df=6-1=5\) degrees of freedom. For a 95% interval, \(t_{0.975,5}=2.571\), giving \(2.00\pm2.571(0.5774)=2.00\pm1.484\), or \((0.516,3.484)\) points. Under the paired t model, this interval excludes zero. The sample provides evidence of a positive mean difference; it does not establish that treatment caused the difference unless the causal assumptions are also credible. The interpretation of a higher score as beneficial or harmful depends on the scale.
For the two-sided test of \(H_0:\mu_D=0\) against \(H_a:\mu_D\ne0\), the statistic is \(t=2.00/0.5774=3.464\), with 5 degrees of freedom, giving a p-value of about 0.018. At \(\alpha=0.05\), the data are inconsistent with the zero-mean-difference model under the paired t assumptions. The p-value is not the probability that the null hypothesis is true.
Why the Paired Standard Error Is Appropriate
A paired estimate is the mean of the observed within-pair differences, not the difference between two unrelated samples. The variance identity explains why this matters: within a pair, shared outcome variation can cancel when one outcome is subtracted from the other. For random variables \(Y_T\) and \(Y_C\) in a pair, the variance of the difference is \(\operatorname{Var}(Y_T)+\operatorname{Var}(Y_C)-2\operatorname{Cov}(Y_T,Y_C)\). When the within-pair covariance is positive, ignoring it can substantially overstate the variability of the paired mean.
Worked Example: An Outlier and a Wide Interval
Worked Example: An Outlier and a Wide Interval
In another hypothetical example, five treated-minus-comparison differences in a continuous outcome are \(1,2,2,3,12\) units. The value 12 is much larger than the other four. The mean is \(\bar{D}=(1+2+2+3+12)/5=4.00\) units, while the median is 2 units. This contrast signals that the mean is strongly influenced by the largest pair difference.
For the paired t calculation, deviations from 4 are \(-3,-2,-2,-1,8\); their squared values sum to \(82\). Therefore, \(s_D=\sqrt{82/4}=4.528\) units and \(SE=4.528/\sqrt{5}=2.025\) units. The test statistic for a zero mean difference is \(t=4.00/2.025=1.975\), with \(df=4\), giving a two-sided p-value of about 0.12. The 95% interval is \(4.00\pm2.776(2.025)\), or approximately \((-1.62,9.62)\) units.
The interval is wide and includes zero. This does not prove there is no average difference. It indicates substantial uncertainty under the model. Before relying on the t result, inspect how the outlying pair arose, verify the source data and outcome measurement, and consider whether the differences are plausibly modeled by a normal distribution. With only five pairs, the data cannot reliably establish that the population differences are approximately normal. Reporting the median and range alongside the mean and standard deviation makes the observed skew or outlier visible, but does not automatically replace the mean-effect estimand with a different one.
Worked Example: Pairing Versus an Independent-Groups Standard Error
Worked Example: Pairing Versus an Independent-Groups Standard Error
Consider four hypothetical matched pairs with comparison outcomes \(10,20,30,40\) and treated outcomes \(11,23,35,47\), all in the same outcome units. The pair differences are \(1,3,5,7\), so \(\bar{D}=4.00\). The deviations from 4 are \(-3,-1,1,3\); hence \(s_D=\sqrt{20/3}=2.582\), and the paired standard error is \(2.582/\sqrt{4}=1.291\) units.
If the two groups are incorrectly treated as independent, their sample variances are \(s_T^2=240\) and \(s_C^2=166.667\). The independent-groups standard error is \(\sqrt{s_T^2/4+s_C^2/4}=\sqrt{(240+166.667)/4}=10.083\) units. The estimated group-mean difference remains 4 units because every treated outcome and every comparison outcome is used once, but the independent calculation discards the pairing and its strong shared baseline pattern. It therefore gives very different uncertainty in this example.
The paired t statistic is \(4.00/1.291=3.098\), with \(df=3\). The 95% interval uses \(t_{0.975,3}=3.182\): \(4.00\pm3.182(1.291)\), or approximately \((-0.108,8.108)\) units. This interval includes zero. The small sample illustrates why a large-looking estimate alone is not enough to establish precision.
Diagnostics, Alternatives, and Limits
Check the analysis at the level of the differences. Plot or tabulate their distribution, inspect the minimum and maximum, and look for outliers, skewness, data errors, or clusters of tied values. Check that pair construction and outcome measurement are correct and that each person appears in only one pair if the analysis assumes independent pairs. If pairs share people, controls are reused, or observations are clustered, ordinary paired t standard errors may not reflect the dependence. A design-appropriate variance method may be needed.
A paired t analysis differs from an independent two-sample analysis because it uses the covariance within pairs. It also differs from an adjusted regression analysis, which models outcomes using covariates and may target or estimate a different quantity. Neither an independent t test nor a regression automatically solves confounding; each requires its own modeling and design justification. For strongly skewed differences or influential outliers, report descriptive summaries and consider a suitable robust or distribution-sensitive analysis. Such alternatives can answer a different question: for example, a signed-rank procedure is not simply a distribution-free test of the mean difference.
The conventional paired t interval usually treats the matched pairs as the analysis sample and conditions on the matching that produced them. It does not automatically account for uncertainty from estimating the propensity score, choosing a matching specification, or selecting a subset of treated people. The practical adequacy of that conventional inference depends on the design and inferential target. State the matching procedure, number and target of retained pairs, difference direction, estimate, uncertainty method, and relevant limitations. Matching can improve measured balance, but it cannot remove unmeasured confounding merely by forming pairs.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Four pair differences are \(-1,0,2,3\) points. Calculate their mean.
- Why does the paired t standard error use the standard deviation of the differences rather than the separate outcome standard deviations?
- For \(m=10\) pairs, what degrees of freedom does the usual paired t procedure use?
- Does a paired t confidence interval that excludes zero establish a causal effect? Give a reason.
Solutions. 1. \(\bar{D}=(-1+0+2+3)/4=1.00\) point. 2. The analysis uses each within-pair contrast; its variability incorporates the covariance between the two outcomes in each pair, which separate-group calculations discard. 3. \(df=m-1=9\). 4. No. Excluding zero provides evidence against a zero mean difference under the paired t model, but causal interpretation also requires assumptions such as consistency, no interference, conditional exchangeability, positivity, and an adequately constructed match.