What Makes a Match Useful?
Matching is a design choice: researchers use measured characteristics to create pairs or groups of participants who are similar in ways that matter for the comparison. In an experiment, the participants in a pair may then be assigned to different treatments. In an observational study, a participant receiving one treatment may be matched to a similar participant receiving another. In either setting, matching is useful only if the chosen characteristics make the compared participants more meaningfully comparable.
A match on age may be valuable when age is related to the outcome or to which treatment a person receives. A match on an arbitrary identification number is not. And a match can be exact yet poor: two people of the same age may differ greatly in baseline health, even when that health characteristic is more important to the outcome.
Assess balance for the variables that matter, using their values and distributions—not simply a record of which variables were named in the matching plan. Compare group means for quantitative characteristics, but also check ranges, spread, and whether one group has unusually high or low values. For categorical characteristics, compare the proportions in each category. Similar averages can conceal different distributions: two groups can have the same mean age but very different age ranges.
The best candidates for matching are generally measured before treatment or exposure and are related to the outcome, the assignment or receipt of treatment, or both. They should also be measured reliably. Matching cannot balance an unmeasured characteristic, and it does not guarantee comparability on every characteristic that was not used.
Why Matching on a Relevant Characteristic Can Help
Suppose an outcome tends to increase with a baseline characteristic such as initial symptom severity. If one group starts with more severe symptoms, its later outcomes may differ partly because of that starting imbalance. Pairing participants with similar initial severity reduces that particular difference in the comparison. The value of a match therefore depends on the characteristic’s relevance, not merely on whether the characteristic can be matched.
A simple model shows why close matching on a relevant baseline characteristic can make a comparison clearer. Let \(Y\) be an outcome, \(T\) indicate the treatment group, and \(Z\) be a baseline characteristic. Suppose, for illustration, that the outcome follows an additive model:
Here \(\beta_2\) represents how the outcome changes with \(Z\), under the model, and \(\varepsilon\) represents other variation. For one matched pair, subtract the comparison participant’s outcome from the treatment participant’s outcome. The shared intercept cancels:
When the pair has one participant in each group, \(T_T-T_C=1\). If they match exactly on \(Z\), then \(Z_T-Z_C=0\), so the term \(\beta_2(Z_T-Z_C)\) also vanishes. If they are only close matches, that term may remain but be small. This is a justification under the stated additive model, not a guarantee: important characteristics may be unmeasured, the model may not describe the outcome well, and matching on one characteristic does not ensure balance on others.
Worked Example: Checking Balance on Relevant Baseline Characteristics
Consider a hypothetical study comparing a program with usual care. Four pairs were formed using participants’ ages and baseline symptom scores. The score is measured on the same scale for everyone, with higher values indicating more symptoms.
| Pair | Program age | Usual-care age | Program baseline score | Usual-care baseline score |
|---|---|---|---|---|
| 1 | 42 | 43 | 18 | 19 |
| 2 | 55 | 54 | 24 | 23 |
| 3 | 61 | 60 | 20 | 21 |
| 4 | 48 | 49 | 16 | 16 |
For each characteristic, calculate the program value minus the usual-care value. The age differences are \(-1, 1, 1, -1\) years. Their mean is
The mean age is also 51.5 years in each group. For the baseline score, the differences are \(-1, 1, -1, 0\) points, with mean
The group means for baseline score are 19.5 points in the program group and 19.75 points in usual care. The individual pair differences are small, and both groups have similar values across these measured characteristics. That supports the conclusion that these four pairs are well balanced on age and baseline symptom score. It does not establish balance on other factors, nor does it show that matching has removed every source of bias.
When the Match Is Too Weak
A weak match can create the appearance of pairing without balancing a characteristic that matters. This can happen when the matching rule focuses on an easy-to-measure feature, uses a broad tolerance, or ignores an important baseline difference. The appropriate question is not simply “Were participants matched?” but “How similar are they on the characteristics important to this comparison?”
Worked Example: Matching Age but Missing a More Important Difference
Suppose two program participants and two usual-care participants are matched by age in a hypothetical symptom study. The program participants are aged 40 and 50, with baseline symptom scores of 8 and 12. The usual-care participants are aged 41 and 49, with scores of 19 and 23. Higher scores mean worse symptoms.
The mean ages are both 45 years:
The age match looks close. But the mean baseline symptom scores are 10 points in the program group and 21 points in usual care:
Thus the program group starts 11 points lower on average. If baseline severity is related to later symptoms, this imbalance matters even though the ages are well matched. Matching on age alone was not enough to make the groups comparable on the measured characteristics that may matter to the outcome. A stronger design would consider baseline severity as well, if it is appropriate to match on and there is sufficient overlap between participants.
Poor matches can also result from measurement problems. For example, matching on a rough self-reported measure may leave substantial differences in the true characteristic. Similarly, a rule that pairs participants just because they fall into the same broad category may hide meaningful variation within that category. Balance should be assessed using the actual measured values and the matching tolerances, not inferred from the label “matched.”
Overmatching on Irrelevant Details
Adding more matching requirements is not automatically an improvement. A variable unrelated to the outcome and to treatment assignment may contribute little to comparability. Requiring exact matches on such a variable can make it difficult to find pairs, exclude otherwise suitable participants, and reduce the amount of information available. It can also make the retained sample less representative of the people the study was intended to describe.
A genuinely irrelevant variable does not, by itself, guarantee bias if researchers match on it. The concern is that the restriction may cost participants and precision without a compensating gain, or that the variable is not truly irrelevant. For example, matching on clinic location could matter if clinics differ in care practices or patient mix, even if location initially seems like an administrative detail. Researchers need a substantive reason for treating a variable as irrelevant.
Worked Example: Exact Matching on an Unhelpful Detail
A hypothetical study has 12 suitable pairs matched on baseline blood pressure and age. A researcher then requires exact matching on the day of the week of each participant’s appointment. This extra requirement leaves only five pairs. Assume, solely to illustrate the precision cost, that the standard deviation of the paired differences remains 10 mmHg under either rule.
With 12 pairs, the standard error of the mean paired difference is
With five pairs, it is
The increase, calculated from the unrounded values, is
Under the example’s assumption that the paired-difference standard deviation is unchanged, the exact weekday rule increases the standard error by about 1.585 mmHg. If weekday has no meaningful connection to the outcome or treatment, this loss of information has no clear benefit. If weekday does matter—for instance, if staffing patterns affect care—then it may be worth considering. The decision should rest on the variable’s role, not on a general preference for more exact matches.
Matching Choices That Can Distort a Comparison
The timing of a matching variable matters. To improve comparability at the start of a study, matching is generally based on characteristics measured before treatment or exposure. Matching on a variable that treatment may have changed can remove part of the difference the study is trying to examine. This is often called overadjustment: the analysis controls for a step on the pathway between treatment and outcome.
Worked Example: Matching on a Post-Treatment Measurement
Imagine a hypothetical program intended to lower blood pressure. The program may reduce blood pressure after several months, and lower blood pressure may in turn reduce the risk of a later health event. Researchers want to compare the program with usual care on that later event. If they match participants on blood pressure measured after the program has begun, the match is based on a characteristic the program may already have changed.
As a result, program participants whose blood pressure was lowered may be paired with usual-care participants who happened to have similarly low measurements for other reasons. The comparison no longer preserves the program’s full effect operating through blood pressure. It instead compares groups made similar on a possible consequence of treatment. If the question concerns the overall difference associated with the program, matching on this post-treatment measurement can therefore distort the comparison. A baseline blood pressure measurement, taken before the program, is a different variable and may be a suitable matching characteristic.
A related danger occurs when matching or selection uses a variable influenced by both treatment and another factor related to the outcome. Conditioning on such a shared consequence can make treatment appear associated with that other factor, even if it was not associated before selection. This is one route to selection bias. The practical safeguard is to map out when each candidate variable was measured and ask what may cause it before using it as a matching criterion.
Matching also has a support limitation: a suitable counterpart may not exist for every participant. Discarding participants without a match can change the population represented by the analysis. The comparison then pertains most directly to the participants who could be matched, rather than automatically to everyone initially considered. Report which participants were excluded and describe how the retained sample differs from the original group.
A Practical Review of a Matching Plan
Before relying on a matched comparison, review the design in a fixed order. This makes it easier to separate balance, efficiency, and bias questions, which are related but not interchangeable.
Confirm that each matching variable was measured before treatment or exposure if the goal is to balance baseline characteristics.
Ask whether each variable plausibly relates to the outcome or treatment assignment, and whether it was measured well enough to match meaningfully.
Compare the groups’ distributions on important characteristics, including means, ranges, categories, and notable extremes where appropriate.
Determine how many participants could not be matched and whether the resulting sample represents a narrower population.
Ask whether restrictive rules waste information, control a possible consequence of treatment, or condition on a variable that could create selection bias.
Common Mistakes
- “The groups have the same mean, so they are balanced.” Equal means can hide different spreads, ranges, or category proportions. Inspect the distributions of the important variables.
- “More matching rules must make a better study.” Extra rules can discard participants and reduce precision. Keep a rule only when its expected benefit justifies its cost.
- “A matched study has no confounding.” Matching addresses only measured characteristics used in the design, and even those may remain imbalanced. Unmeasured differences can remain as well.
- “Any measured variable is safe to match on.” A variable measured after treatment may be a consequence of treatment. Matching on it can remove part of the difference of interest or create selection bias.
- “A good match proves the treatment caused the outcome.” Balance is a design feature, not by itself proof of a causal effect. The study’s design and remaining sources of bias still matter.
Check Your Understanding
Use the ideas in this tutorial to answer the following questions.
- Why can two groups with the same mean age still differ meaningfully in age balance?
- A study matches on age but not on a baseline characteristic strongly related to the outcome. What should be checked before concluding that the match is adequate?
- Why might exact matching on a variable unrelated to treatment and outcome be inefficient, even if it does not automatically create bias?
- Explain why matching on a measurement taken after treatment begins can change the comparison a study makes.
- What information should be reported about participants who could not be matched?