Why the Scale of a Caliper Matters
A caliper is a rule that declares a treated–control pair eligible only when the distance between their propensity scores is no greater than a specified width. The width has meaning only together with the scale on which distance is measured. In particular, applying a cutoff to the raw propensity scores is not equivalent to applying the same numerical cutoff to their logits.
This tutorial focuses on calculating and interpreting distance on the logit propensity-score scale. It does not choose a universally best width: that choice depends on the analysis and is addressed in the earlier material on choosing a caliper width. The practical point here is to use the selected width on the scale for which it was selected, and to calculate that scale’s distance accurately.
The logit is the natural logarithm of the estimated odds of treatment. A propensity score of 0.60, for example, has estimated odds \(0.60/(1-0.60)=1.5\) and logit \(\log(1.5)\). Logit distances are therefore differences in log odds, not differences in probabilities. They are dimensionless, but their numerical values are on a different scale from raw-score gaps.
The estimates being compared should come from the same fitted propensity-score model and be on the same score scale. As with other propensity-score matching procedures, a caliper is a design restriction, not a guarantee of causal identification: its use does not by itself establish that measured covariates are balanced or that the assumptions needed for causal interpretation hold.
Calculating the Logit Distance
For two scores \(p_i\) and \(p_j\) strictly inside the unit interval, it is often convenient to calculate the distance directly from their odds:
The second equality follows from the logarithm rule \(\log(a)-\log(b)=\log(a/b)\). It can be useful for hand calculations, but it does not change the quantity being measured: it is still the absolute difference between logits. Keep full precision through the logarithms and subtraction, then round the final distance. Subtracting logits that have already been rounded can change the last reported digit.
Worked Example: One Pair, Two Different Caliper Decisions
Suppose a hypothetical treated unit has estimated propensity score 0.60 and a hypothetical control unit has score 0.65. Compare their eligibility under a raw-score cutoff of 0.10 and a logit-scale cutoff of 0.10. The equal numerical cutoffs are used only to illustrate that the scales differ.
Raw-score distance. The absolute score gap is \( |0.65-0.60|=0.05 \). Since \(0.05\leq0.10\), the pair is eligible under the raw-score rule.
Logit distance. Using the exact scores, $$ \left|\log\left(\frac{0.65}{0.35}\right) -\log\left(\frac{0.60}{0.40}\right)\right| =\left|\log\left(\frac{26}{21}\right)\right| =0.213574\ldots\approx 0.2136. $$ Since \(0.2136>0.10\), the pair is not eligible under the logit-scale rule.
The two decisions do not conflict: they apply the same numerical threshold to different quantities. A raw gap of 0.05 is a probability difference; a logit distance of about 0.2136 is a difference in log odds.
Why Equal Raw Gaps Need Not Mean Equal Logit Distances
The logit transformation is nonlinear. Its derivative at a score \(p\) is
This derivative is smallest at \(p=0.5\) and increases as \(p\) approaches either 0 or 1. More precisely, for \(0<p<q<1\),
The raw gap \(q-p\) is the length of the interval of integration, while the integrand gives different weight to different parts of the score range. Thus, the same raw gap can produce a larger logit distance near the boundaries than near the center. This is the mathematical reason a fixed raw-score cutoff cannot be translated into a single, score-independent logit cutoff.
Worked Example: Equal Raw Gaps at Different Score Levels
Consider two hypothetical treated–control pairs. The first has scores 0.10 and 0.15; the second has scores 0.80 and 0.85. Each pair has a raw gap of 0.05. Apply a logit caliper of width \(c=0.40\).
Pair 1. The logit distance is $$ \left|\log\left(\frac{0.15}{0.85}\right) -\log\left(\frac{0.10}{0.90}\right)\right| =\log\left(\frac{27}{17}\right) =0.462624\ldots\approx0.4626. $$ This exceeds 0.40, so the pair is not eligible.
Pair 2. The logit distance is $$ \left|\log\left(\frac{0.85}{0.15}\right) -\log\left(\frac{0.80}{0.20}\right)\right| =\log\left(\frac{17}{12}\right) =0.348307\ldots\approx0.3483. $$ This is no greater than 0.40, so the pair is eligible.
Although both raw gaps equal 0.05, their logit distances differ. A rule based on the logit scale therefore treats these pairs differently at the specified width. The eligibility decisions follow from the calculated distances, not from the raw gaps.
Checking Eligibility Against a Logit Caliper
For each proposed pair, calculate both logits from the unrounded estimated scores, take the absolute difference, and compare that distance with the chosen logit width. The inequality is inclusive: a pair exactly at the width is eligible under the rule \(d_{ij}^{(\operatorname{logit})}\leq c\). In practice, retain more precision than is displayed so that a rounded value near the cutoff does not determine the decision incorrectly.
The next example checks several possible controls against one treated unit. This is a feasibility calculation only; it does not select a nearest neighbor or prescribe a matching algorithm.
Worked Example: Checking Several Candidate Controls
Suppose a hypothetical treated unit has score 0.60. Three hypothetical controls have scores 0.55, 0.56, and 0.65. Use a logit caliper of width \(c=0.20\), and determine which pairs meet the restriction.
The treated unit’s logit is \(\log(0.60/0.40)=\log(3/2)=0.405465\ldots\). For the control with score 0.55, the distance is $$ \left|\log\left(\frac{0.55}{0.45}\right)-\log\left(\frac{0.60}{0.40}\right)\right| =\left|\log\left(\frac{11}{9}\right)-\log\left(\frac{3}{2}\right)\right| =0.204794\ldots\approx0.2048. $$ This is greater than 0.20, so that pair is ineligible.
For the control with score 0.56, the distance is $$ \left|\log\left(\frac{0.56}{0.44}\right)-\log\left(\frac{0.60}{0.40}\right)\right| =\left|\log\left(\frac{14}{11}\right)-\log\left(\frac{3}{2}\right)\right| =0.164303\ldots\approx0.1643. $$ This is no greater than 0.20, so that pair is eligible.
For the control with score 0.65, the distance is $$ \left|\log\left(\frac{0.65}{0.35}\right)-\log\left(\frac{0.60}{0.40}\right)\right| =\log\left(\frac{26}{21}\right) =0.213574\ldots\approx0.2136. $$ This also exceeds 0.20, so that pair is ineligible. Of these three candidates, only the control with score 0.56 satisfies the logit-caliper restriction.
Assumptions, Diagnostics, and Failure Modes
The calculation requires estimated scores strictly between 0 and 1: at either endpoint, the logit is infinite or undefined. It also presumes that the score estimates are on a common scale and that the selected width \(c\) is explicitly a logit-scale width. The caliper calculation itself does not require a normally distributed logit or any particular distribution of scores.
Several checks help reveal when the rule is being misapplied or when the score model creates practical problems:
- Check the scale label. Confirm whether the software or analysis specification means a raw-score or logit-score width. A value such as 0.20 has no unambiguous meaning without that label.
- Check the transformation and the distance. The required quantity is the absolute difference of the two logits. It is not the logit of the absolute score gap, nor the absolute difference of the raw scores.
- Check for boundary estimates. Scores extremely close to 0 or 1 have extreme logits, and exact 0 or 1 estimates do not have finite logits. Investigate the score model and its predictions rather than silently substituting an arbitrary value or clipping scores without a justified procedure.
- Check precision near the cutoff. If the unrounded distance is 0.20004, displaying 0.2000 does not make the pair eligible at \(c=0.20\). Make the decision using the unrounded calculation.
- Check overlap and balance after applying the rule. A distance calculation only establishes whether a pair meets this restriction. It does not demonstrate adequate covariate balance or ensure that all units have comparable counterparts.
These diagnostics complement the earlier material on diagnosing separation and overfit in a score model. Extreme or unreliable estimated scores can make logit distances large or unstable. Conversely, changing the caliper or clipping extreme scores merely to create more eligible pairs can hide a modeling or overlap problem rather than resolve it.
How the Logit Caliper Differs From Alternatives
| Rule | Distance compared with its width | Key implication |
|---|---|---|
| Raw-score caliper | \(|\hat e_i-\hat e_j|\) | Limits a probability-scale gap directly; the same raw gap is treated alike wherever it occurs. |
| Logit-score caliper | \(|\operatorname{logit}(\hat e_i)-\operatorname{logit}(\hat e_j)|\) | Limits a log-odds gap; a fixed raw gap can produce different distances across the score range. |
| No caliper | No distance cutoff | Does not exclude a proposed pair on the basis of a maximum propensity-score distance. |
A raw-score caliper is direct to explain in probability units, but it imposes a uniform maximum probability gap. A logit caliper instead measures separation in log odds and gives more distance to equal-sized raw gaps toward the score boundaries. Neither scale automatically produces good matches: the choice must be coherent with the intended design, and balance and overlap still need to be assessed.
A common mistake is to take a width chosen for the logit scale and apply it directly to the raw propensity scores because both are numbers between roughly zero and one in typical examples. That reasoning is wrong: a logit distance is not a probability difference, and logits are not restricted to the interval from zero to one. The correction is to name the scale explicitly, transform each score if the rule is on the logit scale, calculate the distance on that scale, and compare it with a width defined on the same scale.
A logit caliper is also a restriction, not a complete matching procedure. It says which pairs are permitted; it does not specify how to choose among eligible pairs. The earlier material on caliper matching as a feasibility rule distinguishes these roles. The subsequent matching decision must use the eligible pairs without changing the meaning of this distance rule.
Check Your Understanding
Answer each question, then compare your work with the solutions below.
- A treated unit has estimated score 0.50 and a control has estimated score 0.60. Calculate their raw-score gap and logit distance. Is the pair eligible at logit width \(c=0.30\)?
- Why can two pairs with the same raw-score gap have different logit distances?
- A pair has a calculated logit distance of 0.125, and the logit caliper width is 0.125. Is it eligible under the rule in this tutorial?
- What is wrong with deciding that a pair passes a logit caliper because its raw-score gap is smaller than the numerical width?
Solutions. (1) The raw gap is \(|0.60-0.50|=0.10\). The logit distance is \(|\log(0.60/0.40)-\log(0.50/0.50)|=|\log(1.5)-0|=0.4055\), which exceeds 0.30; the pair is ineligible. (2) The derivative of the logit is \(1/[p(1-p)]\), so a given raw interval receives different logit-scale distance depending on its location. (3) Yes. The rule is \(d_{ij}^{(\operatorname{logit})}\leq c\), so equality qualifies. (4) The raw gap and logit distance are different quantities. Transform both scores, calculate their absolute logit difference, and compare that result with the logit-scale width.