Why Caliper Width Is a Design Choice
A caliper sets the largest distance allowed between a treated unit and a control unit for them to be eligible to match. Choosing its width is therefore a design decision: a narrow threshold rules out more distant pairs, but may leave fewer feasible options; a wide threshold preserves more options, but permits pairs that are less similar in estimated propensity score. There is no width that is best for every data set and every analysis goal.
This tutorial considers distances on the propensity-score probability scale. For treated unit \(i\) and control unit \(j\), let \(\hat e_i\) and \(\hat e_j\) be their estimated propensity scores. Their absolute score distance is \(d_{ij}=|\hat e_i-\hat e_j|\). A proposed width \(c\) makes the pair feasible when \(d_{ij}\leq c\). The choice of \(c\) should be assessed against the actual candidate-pair distances and the design's goals, not chosen by looking at outcome differences.
A caliper restricts which pairs may be used; it does not by itself determine which feasible pairs will be selected. The particular matching procedure, including whether matching is with or without replacement and how it resolves competing choices, can affect the completed match. Thus the number of feasible pairs is informative, but it is not automatically the number of matched pairs.
What Changes When the Width Changes?
If \(c_1\leq c_2\), every pair feasible at \(c_1\) is also feasible at \(c_2\). In set notation, \(\mathcal{F}(c_1)\subseteq\mathcal{F}(c_2)\). This gives a useful sensitivity comparison: increasing the width can add feasible options but cannot remove an option under the same distance definition and candidate units.
The statement is about feasibility, not necessarily the exact units selected by a matching procedure. For instance, a greedy procedure may use a newly available pair and thereby change later choices. A wider width can therefore alter the final matched sample even though the original feasible pairs remain allowed. The general design implication is more modest: wider widths loosen the distance restriction, while narrower widths impose it more stringently.
A Practical Way to Compare Candidate Widths
Begin by specifying the estimand and the matching design, then examine the score distances among the candidate treated-control pairs. Compare a small set of plausible widths, rather than choosing a threshold solely because it is conventional or produces a preferred result. For each width, record at least:
- how many candidate pairs meet the threshold, while distinguishing eligible pairs from completed matches;
- the distances among the pairs actually used, including the largest distance;
- how many treated units can be matched under the chosen procedure, if retaining treated units is part of the design goal; and
- covariate balance after matching, evaluated using appropriate pre-treatment covariates.
A useful choice keeps pair distances acceptably restricted while retaining enough feasible matches for the analysis. The relevant balance is not merely “more matches versus fewer matches.” If a narrow width yields strong similarity but leaves many treated units without controls, it may not serve an estimand that targets the full treated group. If a wide width retains nearly everyone but permits weak comparisons, its larger matched sample does not by itself make the design better. State the intended priority and report the consequences of the selected width.
Specify the population or effect the analysis is intended to address, and the matching structure. The choice of width can change which units contribute to the matched analysis.
Summarize the distances available before applying the width. Look for gaps or clusters that make some thresholds substantially more restrictive than nearby values.
For each width, apply the same feasibility rule and matching design; record eligible pairs, completed matches, distance summaries, and post-match covariate balance.
Select a width that reflects the design priorities, and report its scale and units. Include sensitivity results for reasonable alternatives when conclusions may depend on the threshold.
The distance distribution can help make the comparison concrete. If many pair distances cluster just above a proposed narrow width, a small increase may add many options. If there is a large gap above the largest distance admitted by a narrow width, widening to just beyond that gap may add no pairs at all. A width need not be tuned to a particular decimal place when the eligible-pair set is unchanged throughout a range.
Worked Example: A Narrow Width Excludes Several Candidate Pairs
Suppose a hypothetical design has four candidate treated-control pairs with propensity-score distances \(0.012\), \(0.028\), \(0.041\), and \(0.067\). Compare widths \(c=0.03\) and \(c=0.05\). These are candidate-pair distances for illustration; they do not imply that all four pairs can be used together in the final match.
At width \(0.03\), apply \(d_{ij}\leq0.03\). The distances \(0.012\) and \(0.028\) qualify, while \(0.041\) and \(0.067\) do not. Thus two of four listed pairs are feasible.
At width \(0.05\), the distances \(0.012\), \(0.028\), and \(0.041\) qualify; \(0.067\) does not. Three of four listed pairs are feasible. Widening the threshold from \(0.03\) to \(0.05\) adds the pair at distance \(0.041\), but it does not make the pair at \(0.067\) eligible.
The wider threshold retains one more option, but allows a larger score discrepancy for that added pair. Whether this trade-off is acceptable requires considering the full candidate structure and the covariate balance achieved by the resulting match. The pair count alone does not settle the choice.
Why Feasibility Grows With Width
The monotonicity of feasible sets follows directly from the threshold rule. Suppose \(c_1\leq c_2\), and take any pair \((i,j)\in\mathcal{F}(c_1)\). By definition, \(d_{ij}\leq c_1\). Since \(c_1\leq c_2\), it follows that \(d_{ij}\leq c_2\), so \((i,j)\in\mathcal{F}(c_2)\). Therefore:
This result justifies comparing widths in order from narrow to wide: the feasible-pair set can only stay the same or expand. It does not prove that every matching algorithm will retain the same treated units, that the number of completed matches must increase under every procedure, or that a wider width improves the analysis. Those conclusions depend on the matching design and the data.
Worked Example: A Wider Caliper May Add Options but Not a New Match
Consider two treated units, \(T_1,T_2\), and two controls, \(C_1,C_2\). The hypothetical candidate distances are:
| Candidate pair | Distance |
|---|---|
| \(T_1,C_1\) | 0.020 |
| \(T_1,C_2\) | 0.060 |
| \(T_2,C_1\) | 0.025 |
| \(T_2,C_2\) | 0.090 |
With \(c=0.03\), the feasible pairs are \((T_1,C_1)\) and \((T_2,C_1)\). Both treated units have at least one feasible control, but if controls cannot be reused, these two candidate pairs compete for the same control. There are two feasible edges, but they cannot both form a two-pair match without reusing \(C_1\).
With \(c=0.07\), \((T_1,C_2)\) is added because \(0.060\leq0.07\); \((T_2,C_2)\) remains excluded because \(0.090>0.07\). The wider threshold provides an alternative control for \(T_1\), so the feasible options have changed. It does not automatically determine which assignments a matching procedure will make.
This small example shows why a simple count of eligible pairs is not enough. Their arrangement across treated and control units matters, as does whether controls may be reused. The threshold comparison identifies what is allowed; the matching procedure determines how allowed pairs are used.
Balance, Diagnostics, and Limits
Propensity-score proximity is a design criterion, not a guarantee of covariate balance. After choosing a width and forming the match, examine balance on the measured pre-treatment covariates used in the analysis. A width that admits many pairs but leaves important covariates poorly balanced should not be defended solely by its retention rate. Conversely, a highly restrictive width is not automatically satisfactory if it leaves a substantively unrepresentative set of treated units.
Check the distance summaries as well as covariate summaries. Report the number of pairs or units retained under the matching design, the distribution of distances among pairs actually used, and the number of treated units that could not be matched when relevant. Compare a few plausible widths to see whether the design and its balance are sensitive to the threshold. If a small change in width causes a large change in retention or balance, that sensitivity is itself important information.
Caliper choice cannot repair a misspecified propensity-score model, create overlap where the observed scores provide little comparable support, or remove bias from unmeasured confounding. A causal interpretation still relies on the relevant causal assumptions, including consistency, conditional exchangeability given measured covariates, and positivity for the target population. Matching changes the analyzed sample and its comparisons; it does not make those assumptions true by definition.
The width is also tied to the distance scale. A numerical threshold on probability-scale distances is not interchangeable with the same numerical threshold on another scale. Report exactly which score and distance scale were used. The focus here is the choice among widths on the stated probability scale; the scale itself is part of the method specification.
Common Pitfalls
- Choosing a width because it gives the largest sample. Retaining more units does not guarantee close comparisons or good covariate balance. Assess both retention and the quality of the comparisons.
- Treating eligible pairs as completed matches. Several candidate pairs may share a unit, or a matching procedure may not select every eligible pair. Report feasible options separately from the final matched sample.
- Assuming a narrower width is always better. A narrow threshold restricts score distances but can sharply limit feasible comparisons and change the population represented by the matched sample.
- Assuming a wider width only adds matches without changing anything else. It expands the set of allowed pairs, which can alter the choices made by the matching procedure and the composition of the matched sample.
- Using one width without checking alternatives. If the retained sample or balance changes markedly across reasonable thresholds, report that dependence rather than presenting the selected width as innocuous.
Worked Example: Choosing Between Two Widths Using Design Diagnostics
In a hypothetical analysis, a team compares widths \(0.02\) and \(0.05\) using the same score estimates and matching design. At \(0.02\), 62 treated units are matched, the largest distance among used pairs is \(0.019\), and the largest absolute standardized covariate difference is \(0.07\). At \(0.05\), 78 treated units are matched, the largest used-pair distance is \(0.048\), and the largest absolute standardized covariate difference is \(0.16\). These values are hypothetical summaries, not universal thresholds for acceptable balance.
The narrower width retains 62 treated units and yields smaller maximum used-pair distance and smaller maximum covariate difference. The wider width retains 16 more treated units, but the largest distance increases by \(0.048-0.019=0.029\) propensity-score points, and the largest reported covariate difference increases by \(0.16-0.07=0.09\).
There is no automatic winner. If the design prioritizes retaining a broader portion of the treated group, the wider width may be considered, but its weaker balance diagnostics must be addressed and reported. If close comparisons and the observed balance are the priority, the narrower width is more consistent with that goal, while its reduced retention must also be disclosed. The choice should be justified in relation to the target population and supported by sensitivity results, not selected by inspecting outcome estimates.
Check Your Understanding
Answer each question using the feasibility rule and design principles described above. Short solutions follow.
- A pair has distance \(0.035\). Is it feasible at widths \(0.035\) and \(0.03\)?
- Explain why increasing a width cannot remove a previously feasible candidate pair.
- There are six eligible pairs at one width. Can you conclude that six pairs will be used in the final match? Why or why not?
- A wider width retains more treated units but worsens measured covariate balance. What information should be considered before choosing between it and a narrower width?
- Why should the width and its distance scale be reported together?
Solutions. (1) At \(0.035\), the pair is feasible because \(0.035\leq0.035\); at \(0.03\), it is not, because \(0.035>0.03\). (2) If \(d_{ij}\leq c_1\) and \(c_1\leq c_2\), then \(d_{ij}\leq c_2\), so the pair remains feasible. (3) No. Candidate pairs can share units, and the matching design determines which eligible pairs can be used together. (4) Consider the target population, the number and composition of units retained, distances among used pairs, covariate balance, and sensitivity across plausible widths. (5) A width is measured in the units of its distance definition; the same number has different meaning on different scales.