Why the Way Pairs Are Formed Matters
A paired study compares outcomes within pairs, but the arithmetic of a within-pair difference does not by itself tell us whether the difference is causal. The crucial question is how treatment was assigned. If a researcher randomly assigns one member of each pair to treatment, the design can support a causal comparison. If treatment was chosen by the participants, clinicians, or circumstances and the researcher later matches treated and untreated people, the comparison remains vulnerable to confounding.
Pairing can make units more comparable on characteristics used to form the pairs. That may improve precision or reduce some sources of bias. But matching is not a substitute for random assignment: people who receive different treatments may still differ in other ways that affect their outcomes.
Potential Outcomes and the Causal Question
For each unit \(i\), let \(Y_i(1)\) denote the outcome that would occur if that unit received treatment, and let \(Y_i(0)\) denote the outcome that would occur if that same unit received control. The individual causal effect is \(Y_i(1)-Y_i(0)\). Only one of these two outcomes is observed for a unit: the other is its counterfactual outcome.
A common target in a randomized paired experiment is the average treatment effect among the units enrolled in the experiment, the mean of their individual causal effects. In a matched observational study where each treated unit is paired with a control, a natural target is often the average effect for the treated units in the matched sample. The comparison must be interpreted in relation to its target: an effect for enrolled participants is not automatically an effect for a broader population.
What Random Assignment Within Pairs Establishes
In a pair of two units, suppose exactly one is randomly assigned to treatment and the other to control. Let the first unit's two potential outcomes be \(Y_1(1)\) and \(Y_1(0)\), and the second unit's be \(Y_2(1)\) and \(Y_2(0)\). Define the observed difference as the treated unit's outcome minus the control unit's outcome. Randomly choosing which unit receives treatment makes either assignment equally likely.
The following derivation shows why the observed difference is unbiased, over the random assignment, for the average of the two units' causal effects. If unit 1 is treated, the observed difference is \(Y_1(1)-Y_2(0)\). If unit 2 is treated, it is \(Y_2(1)-Y_1(0)\). Therefore:
The second line is the average of the two individual causal effects. With several pairs, the mean of the observed treated-minus-control differences is, over the allowed random assignments, an unbiased estimator of the average treatment effect for the units in those pairs. Pairing does not reveal each person's two potential outcomes; rather, random assignment makes the comparison fair on average across the assignment process.
Worked Example: A Randomized Comparison Within Four Pairs
In a hypothetical experiment, researchers form four pairs of similar training groups. Within each pair, a coin flip assigns one group to a new training program and the other to the standard program. The outcome is a score, in points, on a skills assessment; larger scores are better. The observed treatment-minus-control differences are 3, 1, 4, and 0 points.
The estimated average effect for the eight groups is the mean paired difference:
Because assignment to the two programs was randomized within each pair, this mean difference estimates the average causal effect of assignment to the new rather than the standard program for these groups, subject to the design conditions above. It does not show that every group improved by 2 points, nor does it alone establish how precisely the effect has been estimated. The coin flips, not just the pairing or the positive mean, are what support a causal comparison.
What Observational Matching Can and Cannot Establish
In an observational matched-pair study, researchers may pair a treated person with an untreated person of similar age, baseline health, or other recorded characteristics. The paired difference describes how the observed outcomes compare within those pairs. For that difference to identify a causal effect, the matched controls must provide a credible estimate of what would have happened to the treated units without treatment.
That claim requires assumptions. In particular, after conditioning on the characteristics used in matching, treatment assignment must be unrelated to the potential outcomes; this is often called conditional exchangeability or no unmeasured confounding. There must also be comparable treated and untreated units at the relevant covariate values, called positivity. Consistency requires that the treatment being studied is well-defined and that each unit's observed outcome corresponds to the treatment it received. As above, interference must be absent or appropriately addressed.
Matching can make measured characteristics more similar, but it cannot demonstrate that all important characteristics have been measured or balanced. If an unmeasured factor affects both treatment choice and outcome, treatment and control units may still differ in their counterfactual outcomes. The paired difference can then mix a treatment effect with confounding.
Worked Example: A Matched Observational Comparison
Suppose a hypothetical observational study matches five people who chose a nutrition program with five people who did not, pairing them on age and baseline weight. The outcome is weight change in kilograms, defined so that a larger value means more weight loss. The five within-pair differences (program participant minus matched control) are 3, 1, -1, 4, and 0 kg.
The mean observed paired difference is:
The defensible direct conclusion is that, in this hypothetical matched sample, program participants lost an average of 1.40 kg more than their matched controls over the study period. Calling this a 1.40 kg causal effect additionally requires that the matching and analysis control all relevant confounders, that comparable controls exist for the treated participants, and that the other assumptions hold. If, for example, motivation affects both choosing the program and weight loss but is not measured, the difference may partly reflect motivation rather than the program.
A Confounder Can Reverse a Matched Comparison
A confounder is a variable related to both treatment assignment and the outcome that can distort their observed relationship. Matching on other characteristics does not remove this distortion if the confounder remains imbalanced. The following hypothetical construction makes the issue visible by specifying potential outcomes, even though a real observational study cannot observe both potential outcomes for each person.
Worked Example: Matching on Age Does Not Control Severity
Imagine three matched pairs with the same age within each pair. In every pair, the person with more severe symptoms chooses treatment and the person with milder symptoms does not. The outcome is symptom improvement in points, with larger values representing more improvement. For illustration, suppose that each severe person would improve by 4 points without treatment and 6 points with treatment, while each mild person would improve by 8 points without treatment and 10 points with treatment.
For a severe treated person, the causal effect is \(6-4=2\) points. For the mild untreated person, the observed outcome is 8 points. Thus, in each pair, the observed treated-minus-control difference is:
Across the three identical pairs, the mean observed difference is also \(-2\) points. Yet the treatment effect for each severe person is \(6-4=2\) points. The negative observed comparison arises because treatment recipients began with greater severity, which is related to improvement in this example. Pairing on age did not balance severity. In actual observational data, the untreated outcome that each treated person would have had is unobserved, so this example's true effects would not be directly available to diagnose the bias.
Limits on the Causal Claim
Even a well-conducted randomized paired experiment supports a bounded conclusion. Random assignment supports a causal effect of the assigned treatment conditions for the studied units; it does not ensure that those units represent a wider population. Recruitment, setting, treatment implementation, and outcome measurement all matter when considering generalizability.
Also distinguish the effect of assignment from the effect of actually receiving treatment. If some participants do not follow their assignment, a direct comparison by randomized assignment still concerns the effect of being assigned to the treatment condition. It does not automatically identify the effect of treatment received, because adherence may be related to participants' characteristics or outcomes.
For observational matching, the central limit is stronger: causal interpretation depends on assumptions about why treatment was received. Similarity on measured matching variables is useful evidence about comparability on those variables, but it is not proof of no unmeasured confounding. A careful report states the observed comparison and the assumptions needed to interpret it causally.
Ask whether a chance mechanism assigned treatment within each pair, or whether treatment choices occurred before matching.
Specify whether the question concerns the enrolled units, treated units in the matched sample, or a wider population.
For randomized assignment, describe the causal effect supported by the design and its scope. For observational matching, describe the paired association and make causal interpretation conditional on the relevant assumptions.
Common Mistake
Wrong reasoning: “The groups were carefully matched, so their outcome difference is the causal effect.” Matching can reduce differences in characteristics used to form pairs, but it does not randomize treatment or guarantee balance on unmeasured confounders.
Correction: First establish whether treatment assignment was randomized. If it was, describe the causal effect of assignment for the studied units, subject to the design conditions. If it was not, report the matched outcome difference as an observational comparison, and explain that a causal interpretation requires no important residual confounding and the other assumptions for causal identification.
Check Your Understanding
Answer each question in terms of the design and the scope of the conclusion.
- A researcher matches patients on age and baseline blood pressure, then compares those who chose a medication with those who did not. Was treatment randomized? What can the paired difference establish directly?
- In a paired experiment, a coin flip assigns one person in each pair to treatment. What feature of the design supports a causal interpretation?
- Give an example of an unmeasured confounder that could remain after matching on age and baseline health.
- What is the difference between the effect of assignment and the effect of treatment received when some participants do not follow their assignments?
- Why does a causal result for enrolled participants not automatically apply to every person in the wider population?