Why Compare the Two Rules?
Nearest-neighbor matching and caliper-restricted nearest-neighbor matching use the same basic idea: pair a treated unit with a control unit whose propensity score is close. The difference is important. Unrestricted nearest-neighbor matching always takes the closest available control, even if that score distance is large. A caliper adds a maximum permitted distance; if no available control is within that limit, the treated unit remains unmatched. Thus, a caliper can avoid particularly distant pairs, but it can also reduce the number of treated units matched.
The examples below use hypothetical propensity scores and a specified treated-unit order. They illustrate a greedy, one-to-one procedure without replacement: once a control is assigned, it cannot be assigned again. Distances are absolute differences on the raw propensity-score scale. This scale and the caliper value must be stated; a numerical caliper on another scale, such as the logit of the propensity score, is a different rule.
A Shared Hypothetical Score Table
Suppose four treated units and five control units have the following estimated propensity scores. All numbers are hypothetical. The stated processing order is \(T1,T2,T3,T4\), and controls are not reused.
| Unit | Group | Propensity score |
|---|---|---|
| T1 | Treated | 0.10 |
| T2 | Treated | 0.15 |
| T3 | Treated | 0.40 |
| T4 | Treated | 0.80 |
| C1 | Control | 0.15 |
| C2 | Control | 0.30 |
| C3 | Control | 0.42 |
| C4 | Control | 0.78 |
| C5 | Control | 0.95 |
The propensity score is the estimated probability of treatment given measured covariates. Matching on it is a design operation, not by itself proof that treatment groups are exchangeable. A causal interpretation of a matched comparison additionally requires, among other conditions, consistency, no interference, conditional exchangeability given measured covariates, and sufficient positivity for the target population. In practice, the score model and resulting matches should be assessed for covariate balance and overlap; a close score alone does not verify these conditions.
Worked Example: Greedy Nearest-Neighbor Matching Without a Caliper
Process \(T1,T2,T3,T4\) in that order. At each step, calculate distances to the controls that have not yet been used, then assign the nearest one.
Step 1, T1. Its distances to all controls are \(|0.10-0.15|=0.05\), \(|0.10-0.30|=0.20\), \(|0.10-0.42|=0.32\), \(|0.10-0.78|=0.68\), and \(|0.10-0.95|=0.85\). The smallest is 0.05, so assign T1 to C1.
Step 2, T2. C1 is no longer available. The distances to C2 through C5 are \(0.15, 0.27, 0.63,\) and \(0.80\), respectively. The smallest is 0.15, so assign T2 to C2.
Step 3, T3. C1 and C2 are used. The distances to C3, C4, and C5 are \(0.02, 0.38,\) and \(0.55\). Assign T3 to C3.
Step 4, T4. C4 and C5 remain. Their distances are \(|0.80-0.78|=0.02\) and \(|0.80-0.95|=0.15\). Assign T4 to C4.
The resulting pairs are T1–C1, T2–C2, T3–C3, and T4–C4. No treated unit is unmatched, and C5 is unused. The pair distances are 0.05, 0.15, 0.02, and 0.02. In particular, T2's match is much farther away than the other three, despite being the nearest control available at its step.
Adding a Caliper
Now impose a raw-score caliper of \(c=0.04\), retaining the same order and no-replacement rule. A control is eligible only when its distance is at most 0.04. The restriction is applied at each step to the controls still available; a control outside the caliper is not assigned merely because it is the nearest.
Worked Example: Caliper-Restricted Matching at 0.04
Step 1, T1. Its nearest control is C1 at distance \(|0.10-0.15|=0.05\). Because \(0.05>0.04\), no available control qualifies. Leave T1 unmatched; no control is consumed.
Step 2, T2. C1 is still available, and \(|0.15-0.15|=0\). Assign T2 to C1. The other controls are farther away.
Step 3, T3. Among the remaining controls, C3 is nearest: \(|0.40-0.42|=0.02\leq0.04\). Assign T3 to C3.
Step 4, T4. C4 is nearest among controls still available: \(|0.80-0.78|=0.02\leq0.04\). Assign T4 to C4.
The caliper result is T2–C1, T3–C3, and T4–C4, with T1 unmatched. C2 and C5 are unused. Compared with unrestricted greedy matching, the change is not simply that a distant pair disappears: leaving C1 unused at T1 allows T2 to take C1 rather than C2. The caliper limits accepted distances but does not guarantee that every treated unit will have a partner.
Worked Example: The Caliper Boundary
Use the same table and order, but set \(c=0.05\). Since the caliper is inclusive, a distance exactly equal to 0.05 is allowed.
For T1, the nearest control is C1 at distance \(|0.10-0.15|=0.05\). Thus \(0.05\leq0.05\), and T1 is assigned to C1. For T2, C1 is now used, so its nearest available control is C2 at distance \(|0.15-0.30|=0.15\), which exceeds 0.05; T2 is unmatched. T3 takes C3 because its distance is 0.02, and T4 takes C4 because its distance is 0.02. The resulting pairs are T1–C1, T3–C3, and T4–C4, with T2 unmatched. C2 and C5 are unused.
This result differs from the 0.04 caliper result: at 0.05, T1 takes C1 first, so T2 cannot use it. The boundary convention and exact caliper value can therefore affect the final allocation.
Greedy Order and Alternative Approaches
A greedy assignment depends on the order in which treated units are processed. This is not a minor computational detail: an early assignment changes which controls remain available later. The following calculation uses the same score table, no replacement, and \(c=0.04\), but processes \(T2\) before \(T1\).
Worked Example: Changing the Treated-Unit Order
Process T2 first. It has an exact match to C1 because \(|0.15-0.15|=0\), so assign T2–C1. Next, T1 cannot use C1. Its nearest remaining control is C2, at distance \(|0.10-0.30|=0.20\), which exceeds 0.04; leave T1 unmatched. T3–C3 and T4–C4 are still allowed at distances 0.02 each. The result is T2–C1, T3–C3, and T4–C4, with T1 unmatched.
This order happens to give the same set of pairs as the earlier 0.04-caliper example, but the identity of the unmatched treated unit changes. With a different score table, changing order can also change the control assigned to a treated unit. A matching analysis should specify its processing order and tie rule rather than treating a greedy result as unique.
The main alternatives answer different design questions. Unrestricted greedy nearest-neighbor matching prioritizes making a pair for each treated unit when a control remains, but can accept a large distance. Caliper-restricted greedy matching forbids distances beyond a stated limit, accepting fewer pairs when necessary. Optimal pair matching instead chooses assignments by optimizing a specified total-distance objective across the set of pairs; constrained optimal matching can also impose eligibility restrictions. These global assignment approaches are not the same as sequential greedy matching. The useful comparison is not that one rule always wins, but whether its trade-off—pair count, distance, and retained population—fits the design objective.
Diagnostics, Interpretation, and Common Pitfalls
After matching, report the number of treated units matched and unmatched, the number of controls used and unused, and the distribution of within-pair distances. For caliper matching, verify that every retained pair satisfies the stated cutoff. Check covariate balance after matching, not only similarity of propensity scores, and inspect whether units excluded by the caliper occupy a particular region of the score distribution. These checks can reveal poor overlap, an overly restrictive caliper, or covariate imbalance that the score-distance rule did not resolve.
A caliper is a design choice, not a guarantee of causal validity. If the caliper discards treated units, the resulting comparison pertains to a matched subset; it need not estimate an effect for all treated units or the full sample. Conversely, avoiding unmatched units by allowing very distant pairs may undermine comparability. The estimand and target population should be made clear, especially when matching changes which units are retained.
- Mistake: “Nearest neighbor means a good match.” The nearest available control can still be far away, as T2's distance of 0.15 shows. Report distances and consider whether a prespecified caliper is appropriate.
- Mistake: “A caliper only removes bad pairs and leaves every other assignment unchanged.” In the 0.04 example, rejecting T1's candidate leaves C1 available for T2. Recompute sequential availability after every decision.
- Mistake: “A matched score proves the measured covariates are balanced.” Similar scores do not certify balance in every covariate. Examine covariate balance directly.
- Mistake: “Unmatched treated units can be ignored without changing the target.” Exclusion can change the population represented by the matched comparison. State which units were retained and what population the analysis addresses.
Check Your Understanding
Answer each question using the procedures and hypothetical scores in this tutorial. Brief solutions follow the questions.
- Under unrestricted matching in the worked table, what distance is assigned to T2, and why is C1 unavailable?
- With a 0.04 caliper and the stated order, which treated units are unmatched, and which controls are unused?
- With a 0.05 inclusive caliper and the stated order, does T1 match C1? Show the comparison that decides.
- Why can processing T2 before T1 change which treated unit is unmatched under the 0.04 caliper?
- Name two diagnostics, beyond counting pairs, that should be considered after matching.
Solutions. (1) T2 is matched to C2 at distance \(|0.15-0.30|=0.15\); C1 was assigned to T1 at the preceding step. (2) T1 is unmatched, while C2 and C5 are unused; T2–C1, T3–C3, and T4–C4 are the retained pairs. (3) Yes. The distance is \(|0.10-0.15|=0.05\), and \(0.05\leq0.05\), so the inclusive rule allows the match. (4) T2 can take C1 at distance zero when processed first. T1 then has only controls at distances greater than 0.04, so T1 rather than T2 is unmatched. (5) Appropriate checks include within-pair score-distance summaries, covariate balance after matching, and overlap or the score distribution of excluded units.