Matched-Pair Risk Differences For A Binary Outcome

Why Compare Outcomes Within Matched Pairs?

In a 1:1 matched study, each treated individual is linked to one comparison individual. For a binary outcome, such as readmission within 30 days, the outcome is recorded as 1 if it occurs and 0 otherwise. The natural comparison is the treated person's outcome minus the matched comparison person's outcome. This preserves the pairing: a treated person with an outcome is contrasted with the outcome of the particular comparison person selected for that pair.

The resulting estimate is a risk difference in the matched sample. It is not automatically an effect for the original study population. A causal interpretation additionally requires assumptions such as consistency, no unmeasured confounding after conditioning on the measured covariates used for matching, and adequate overlap for the population represented by the matched pairs. Matching and good measured-covariate balance do not establish those assumptions.

Definition: For \(n\) matched pairs, let \(Y_{Ti}\) be the binary outcome for the treated individual and \(Y_{Ci}\) the outcome for the comparison individual in pair \(i\). Define the pair difference \(D_i=Y_{Ti}-Y_{Ci}\). The matched-pair risk difference is the mean pair difference, \(\widehat{\Delta}_{M}=\bar{D}=\frac{1}{n}\sum_{i=1}^{n}D_i\). It estimates the treated risk minus the comparison risk in the matched sample, in percentage points when multiplied by 100.

Each \(D_i\) can be \(-1\), 0, or 1. A value of 1 means the outcome occurred for the treated individual but not the comparison individual; \(-1\) means it occurred for the comparison individual but not the treated individual. Concordant pairs, in which both outcomes are 1 or both are 0, have \(D_i=0\). They contribute no net difference to the point estimate, although they remain part of the sample of pairs.

Estimating the Risk Difference From Pair Counts

A compact way to calculate the estimate is to count the two types of discordant pair. Let \(b\) be the number of pairs with \(Y_T=1,Y_C=0\), and let \(c\) be the number with \(Y_T=0,Y_C=1\). The sum of all pair differences is \(b-c\), so the estimate is \((b-c)/n\). Equivalently, subtract the comparison group's observed risk from the treated group's observed risk. Both calculations give the same answer when every pair contributes one treated and one comparison individual.

Worked Example: Calculating the Matched-Sample Risk Difference

Suppose a hypothetical study forms 100 distinct treated-comparison pairs and records whether each person is readmitted within 30 days. The pair table has 14 pairs with both outcomes equal to 1, 54 with both equal to 0, 22 with treated outcome 1 and comparison outcome 0, and 10 with treated outcome 0 and comparison outcome 1. The counts sum to \(14+54+22+10=100\).

There are \(14+22=36\) treated individuals with readmission and \(14+10=24\) comparison individuals with readmission. Thus the risks are \(36/100=0.36\) and \(24/100=0.24\), respectively, and their difference is \(0.36-0.24=0.12\).

Using the within-pair contrast gives the same result: \(b=22\), \(c=10\), and \(n=100\), so \(\widehat{\Delta}_{M}=(22-10)/100=0.12\). The estimated matched-sample risk is 12 percentage points higher in the treated group. This is a description of the observed matched sample; a causal explanation requires additional assumptions.

A Confidence Interval That Keeps the Pairing

Pairs are the units whose differences are averaged. If distinct pairs can reasonably be treated as independent draws from the target matched-pair population, estimate uncertainty from the observed variation in the \(D_i\)'s. Within-pair outcome dependence is not a violation of this condition: the two outcomes in a pair are deliberately combined into one difference. The independence condition concerns different pair differences.

Let \(s_D\) be the sample standard deviation of the pair differences. An approximate large-sample 95% confidence interval is \(\bar{D}\pm 1.96SE(\bar{D})\), where \(SE(\bar{D})=s_D/\sqrt{n}\). For binary outcomes, the variance can be computed from the two discordant counts without listing every \(D_i\). Concordant pairs have squared difference zero, while each discordant pair has squared difference one.

Paired large-sample confidence interval: If the pairs are independent and the sample is large enough for the mean pair difference to be approximately normal, use \(\widehat{\Delta}_{M}\pm z_{1-\alpha/2}SE(\widehat{\Delta}_{M})\), where \(SE(\widehat{\Delta}_{M})=\sqrt{\frac{(b+c)-(b-c)^2/n}{n(n-1)}}\). For a 95% interval, \(z_{1-\alpha/2}=1.96\). This is an approximate interval; it can perform poorly when there are very few discordant pairs or the number of independent pairs is small.

The variance expression follows directly from the sample variance of the pair differences. Since \(\sum_i D_i=b-c\) and \(\sum_i D_i^2=b+c\), the sum of squared deviations from the mean is \(\sum_i D_i^2-(\sum_i D_i)^2/n\). Dividing this by \(n-1\) gives \(s_D^2\), and dividing by \(n\) gives the estimated variance of the mean.

s_D^2 = \frac{\sum_{i=1}^{n}(D_i-\bar{D})^2}{n-1} = \frac{(b+c)-(b-c)^2/n}{n-1}, \qquad SE(\bar{D})=\sqrt{\frac{s_D^2}{n}}

This calculation is the key adjustment for pairing. It estimates uncertainty in the average of the observed pair differences, rather than treating the two outcome groups as unrelated samples. In a large sample, the interval has approximately its stated repeated-sampling coverage under the independence and approximation conditions. It does not mean there is a 95% probability that the fixed matched-sample risk difference lies in this particular interval.

Worked Example: Constructing a Paired Confidence Interval

Use the hypothetical 100-pair readmission data above: \(b=22\), \(c=10\), and \(\widehat{\Delta}_{M}=0.12\). There are \(b+c=32\) discordant pairs. The estimated variance of the pair differences is

\(\displaystyle s_D^2=\frac{32-(22-10)^2/100}{99}=\frac{32-1.44}{99}=\frac{30.56}{99}=0.3087.\)

Therefore \(SE(\widehat{\Delta}_{M})=\sqrt{0.3087/100}=\sqrt{0.003087}=0.05556\). The approximate 95% interval is

\(\displaystyle 0.12\pm1.96(0.05556)=0.12\pm0.1089,\)

or approximately \((0.0111,0.2289)\). In percentage points, this is an estimated increase of 12.0 points, with an approximate 95% confidence interval from 1.11 to 22.9 points. The interval excludes zero, but that fact does not by itself show that matching removed unmeasured confounding or that the estimate applies beyond the matched sample.

Assumptions and Diagnostics for Dependence

The paired standard error is appropriate only when the pair differences provide independent units of information, or when the analysis otherwise accounts for their dependence. A matching design creates dependence within each pair, which is handled by calculating one \(D_i\) per pair. But pairing does not automatically remove dependence between pairs. For example, reusing the same comparison individual across several pairs makes those pair differences share an outcome.

Before reporting an interval, check how the matched data were constructed and retained. Confirm that each intended pair has one treated and one comparison individual, that the outcome is coded consistently, and that pair identifiers link the correct individuals. Verify whether any person appears in multiple pairs, whether individuals are clustered in a shared setting such as a clinic, and whether any repeated observations remain. If these features create dependence across pairs, the simple standard error above does not account for it.

Check the pair structure.
Count individuals and pair identifiers. Confirm one treated and one comparison unit per pair for this calculation, and identify any reused individuals or incomplete pairs.

Inspect within-pair outcome patterns.
Tabulate both concordant and discordant pairs. Recalculate \(b\), \(c\), and \(n\) from the pair-level records and check that the implied group risks agree with the reported risk difference.

Assess independence across pairs.
Look for shared individuals, repeated measurements, or common clusters. If pair differences are dependent, use an inference method that accounts for the actual dependence structure rather than applying the independent-pair standard error.

Check whether the approximation is credible.
Very few discordant pairs or few independent pairs can make a normal interval unreliable. Report the discordant counts and consider a method designed for sparse paired binary data.

Worked Example: Detecting Reused Comparison Individuals

Suppose six nominal pairs have contrasts \(D=(1,1,0,0,-1,0)\). The point estimate is \(\bar{D}=(1+1+0+0-1+0)/6=1/6=0.1667\). If these were six independent pairs, the sample variance would be \([3-(1)^2/6]/5=17/30=0.5667\), and the usual standard error would be \(\sqrt{0.5667/6}=0.3073\).

Now suppose inspection of the pair identifiers shows that the same comparison individual was reused in three of these pairs. The six contrasts are then not independent: each of those three contains the same person's outcome. The calculated \(0.1667\) still describes the average of these six recorded pair contrasts, but the standard error \(0.3073\) is not justified by the independent-pair formula. The repeated individual must be accounted for using an analysis appropriate to the dependence and the intended weighting of the matched sample. Treating the six rows as six independent pairs would overstate the amount of independent information.

Choosing and Interpreting the Analysis

For 1:1 matching with a binary outcome, the paired risk difference directly answers how far apart the observed risks are in the matched sample. An independent two-proportion calculation is not a substitute for the paired calculation: it ignores which individuals were matched and uses a variance formula that assumes independent observations across the groups. Conversely, a paired analysis is not justified merely because a data set has a pair label; the records must represent genuine matched units, and dependence between pairs still needs attention.

When the approximation is adequate, report the risk difference, its confidence interval, and the discordant-pair counts. For sparse discordance or limited numbers of independent pairs, a large-sample Wald interval may have poor coverage; use an interval method intended for paired binary data and explain the method used. A p-value from a paired test is not itself a confidence interval for the risk difference, and a test focused on discordance should not be reported as if it changed the estimand.

The estimate weights every pair equally. Thus its target is the average treated-minus-comparison contrast across the retained pairs, not necessarily the effect in all treated individuals before matching or in the full source population. Also, the standard interval described here quantifies sampling uncertainty conditional on the matched-pair analysis as specified; it generally does not capture every source of uncertainty from estimating propensity scores, selecting a matching specification, or choosing which units to retain.

Common mistake: “The outcomes are paired, so the two observations in each pair must be independent.” This reverses the logic. Outcomes within a pair may be associated, and the pair contrast incorporates that association. The standard error instead requires independent pair contrasts, or another method that explicitly accounts for dependence across pairs.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In 80 pairs, 18 have treated outcome 1 and comparison outcome 0, while 6 have treated outcome 0 and comparison outcome 1. What is the matched-pair risk difference?
Why do concordant pairs affect the standard error calculation even though they contribute zero to the point estimate?
What independence condition is needed for the paired standard error: independence of the two outcomes within each pair, or independence across pair differences?
Give one reason a large-sample Wald interval may be unreliable in a matched binary-outcome study.
Does a positive matched-sample risk difference alone establish a causal increase in risk? Explain briefly.

Solutions: (1) \((18-6)/80=0.15\), an estimated 15-percentage-point increase in the treated group. (2) They are included in \(n\), and the variance is computed across all pair differences; their zero differences affect the variation around the mean. (3) The condition is independence across pair differences. Dependence within a pair is handled by forming its single contrast. (4) For example, very few discordant pairs or few independent pairs can make the normal approximation poor. (5) No. Causal interpretation also requires appropriate causal assumptions, including no unmeasured confounding after conditioning on the matched covariates, as well as a clearly defined target population.