Two Criteria for Choosing a Control
A propensity-score caliper and Mahalanobis distance answer different matching questions. The caliper asks whether a control is close enough to a treated unit in estimated treatment propensity to be considered. Mahalanobis distance then ranks the controls that pass that restriction according to their values on selected covariates. The resulting method is a restricted search followed by a refinement; it is not a single distance that combines the score and covariates.
Let \(A_i=1\) indicate that unit \(i\) is treated and \(A_i=0\) indicate that it is a control. Let \(X_i\) denote pretreatment covariates and \(e(X_i)=P(A_i=1\mid X_i)\) the propensity score. In practice, the score is usually estimated; write the estimated score for unit \(i\) as \(\hat e_i\). Let \(Z_i\) be the vector of selected pretreatment covariates used for refinement. These may be a subset of \(X_i\), especially covariates whose balance or precision is a priority.
A common choice is a width defined as a fraction of the sample standard deviation of the estimated logit scores, but the width must be stated and justified for the design. The examples below use explicitly stated widths in logit units. A propensity score of exactly 0 or 1 has no finite logit, so such values require attention to the estimation and overlap problem rather than direct use in this calculation.
Why Restrict First and Refine Second?
Matching only on the propensity score can make scores close while leaving noticeable differences in particular covariates. Conversely, matching only on covariates can choose a control whose overall treatment propensity is far from the treated unit’s. The hybrid rule protects against the latter by excluding controls outside the caliper, then uses the selected covariates to choose among those that remain.
The order matters. A control with the smallest Mahalanobis distance is not chosen if it fails the propensity-score restriction. Likewise, a control that passes the caliper is not automatically chosen if another eligible control is closer on the selected covariates. This method does not promise that a feasible match exists for every treated unit.
Worked Example: Apply the Caliper, Then Compare Covariates
Suppose one treated unit has estimated score \(0.45\). Three controls have estimated scores \(0.47\), \(0.55\), and \(0.43\). Use a logit-score caliper of \(0.20\) logit units. For refinement, the selected covariates are blood pressure and age. The treated unit has values \((140,60)\); the controls with scores \(0.47\) and \(0.43\) have values \((145,62)\) and \((142,64)\), respectively. Suppose the covariance matrix used for these two covariates is \(S=\begin{pmatrix}100&0\\0&25\end{pmatrix}\).
First calculate the treated unit’s logit score: \[ \operatorname{logit}(0.45)=\log(0.45/0.55)\approx -0.2007. \] For the control with score \(0.47\), \[ \left|\log(0.47/0.53)-\log(0.45/0.55)\right| \approx |-0.1201-(-0.2007)|=0.0805. \] This control is eligible. For the control with score \(0.55\), \[ \left|\log(0.55/0.45)-\log(0.45/0.55)\right| \approx |0.2007-(-0.2007)|=0.4013, \] so it is ineligible. The control with score \(0.43\) has logit difference approximately \(0.0812\), so it is also eligible.
Now calculate Mahalanobis distances only for the two eligible controls. For the score-\(0.47\) control, the covariate difference is \((-5,-2)\), giving \[ d_M=\sqrt{\frac{(-5)^2}{100}+\frac{(-2)^2}{25}} =\sqrt{0.25+0.16}=\sqrt{0.41}\approx 0.6403. \] For the score-\(0.43\) control, the difference is \((-2,-4)\), giving \[ d_M=\sqrt{\frac{(-2)^2}{100}+\frac{(-4)^2}{25}} =\sqrt{0.04+0.64}=\sqrt{0.68}\approx 0.8246. \] The score-\(0.47\) control is selected: it passes the caliper and has the smaller Mahalanobis distance. The score-\(0.55\) control cannot be selected, even if its covariates had been closer, because it failed the eligibility rule.
What Mahalanobis Distance Contributes
Mahalanobis distance scales covariate differences by their covariance structure. A difference of five units on a variable that naturally varies by ten units should not automatically count as more important than a difference of two units on a variable that varies by one unit. The matrix \(S^{-1}\) accounts for scale; its off-diagonal entries also account for correlation among selected covariates.
For a short justification, suppose a nonsingular linear transformation changes the covariate coordinates to \(W_i=BZ_i\), where \(B\) is invertible. The covariance matrix in the new coordinates is \(BSB^\mathsf{T}\). The transformed squared distance is \[ (B\Delta Z)^\mathsf{T}(BSB^\mathsf{T})^{-1}(B\Delta Z) =\Delta Z^\mathsf{T}B^\mathsf{T}(B^\mathsf{T})^{-1}S^{-1}B^{-1}B\Delta Z =\Delta Z^\mathsf{T}S^{-1}\Delta Z, \] where \(\Delta Z=Z_i-Z_j\). Thus, with a consistently transformed covariance matrix, the distance does not depend on a nonsingular change of coordinates. This scale adjustment does not decide which covariates are scientifically important; that remains a design choice.
Worked Example: Correlation Changes the Ranking
Suppose both controls below pass a treated unit’s propensity-score caliper. There are two selected covariates, and the covariance matrix is \(S=\begin{pmatrix}4&1\\1&1\end{pmatrix}\). The covariate difference for control A is \((1,1)\), and for control B it is \((2,0)\). Determine which control is closer.
The determinant of \(S\) is \(4(1)-1(1)=3\), so \[ S^{-1}=\frac{1}{3}\begin{pmatrix}1&-1\\-1&4\end{pmatrix}. \] For control A, the squared distance is \[ (1,1)S^{-1}(1,1)^\mathsf{T} =\frac{1}{3}(1-2+4)=1, \] and hence \(d_M=1\). For control B, it is \[ (2,0)S^{-1}(2,0)^\mathsf{T} =\frac{1}{3}(4)=\frac{4}{3}, \] so \(d_M=\sqrt{4/3}\approx1.155\). Control A is closer under Mahalanobis distance. If the off-diagonal covariance had been ignored, the ranking would be based on a different distance; accounting for covariance is part of the stated refinement rule.
Eligibility, Unmatched Units, and Diagnostics
The caliper can leave a treated unit with no eligible control. That is a design outcome, not a reason to silently choose a control outside the stated restriction. The analyst should report how many treated units are unmatched and examine whether the retained treated units differ systematically from those excluded. If the target is the treatment effect for all treated units, dropping treated units may change the population to which the estimate pertains.
After matching, assess balance on the covariates used for refinement and on other important pretreatment covariates. Compare distributions as well as means where relevant, and examine overlap in propensity scores. A small Mahalanobis distance for each selected pair is not proof of overall balance: pairwise closeness on a chosen subset does not guarantee balance on omitted covariates, higher-order features, or the full score distribution. Diagnostics should also check whether the covariance matrix is ill-conditioned or singular, whether ties occur, and whether the results depend strongly on the chosen caliper or covariate set.
Worked Example: A Treated Unit With No Eligible Control
Two treated units have estimated scores \(0.30\) and \(0.70\). Two controls have scores \(0.31\) and \(0.42\). Use a logit-score caliper of \(0.15\). For the treated unit with score \(0.30\), the logit is \(\log(0.30/0.70)\approx-0.8473\). The control with score \(0.31\) has logit \(\log(0.31/0.69)\approx-0.8001\), a difference of about \(0.0472\), so it is eligible. The control with score \(0.42\) has logit \(\log(0.42/0.58)\approx-0.3228\), a difference of about \(0.5245\), so it is ineligible. Mahalanobis refinement may compare the selected covariates for the eligible control, but there is only one candidate.
For the treated unit with score \(0.70\), the logit is \(\log(0.70/0.30)\approx0.8473\). Its differences from the two control logits are approximately \(1.6474\) and \(1.1701\), both greater than \(0.15\). It has no eligible control and remains unmatched under this rule. Relaxing the caliper would change the design and must be justified; the nearest available control is not automatically acceptable merely because no closer candidate exists.
Comparison and Common Pitfall
| Approach | What determines the match? | Important limitation |
|---|---|---|
| Propensity-score matching alone | Closeness of estimated scores | Similar scores do not ensure close values on every particular covariate in finite samples. |
| Mahalanobis matching alone | Distance on the chosen covariates | It does not impose a propensity-score restriction unless one is added. |
| Propensity-score restriction with Mahalanobis refinement | Score caliper defines eligible controls; covariate distance ranks them | May leave units unmatched, and its result depends on both the caliper and the selected covariates. |
Common mistake: “The control with the smallest Mahalanobis distance is always the best match, so it should be selected even if it is outside the propensity-score caliper.” This reverses the rule. The caliper is a feasibility condition, not another preference to trade off against Mahalanobis distance. First remove ineligible controls; then compare distances among the remaining candidates. A second mistake is to treat the smallest distance as evidence that exchangeability holds. Distance describes observed covariate proximity under the chosen design; it cannot measure unobserved confounding.
The method is useful when a score restriction is important for avoiding poor-propensity comparisons, while selected covariates need closer within-caliper alignment. It is not automatically superior to either alternative. Its performance depends on adequate overlap, a defensible score model, a meaningful covariate set, and post-match diagnostics. The caliper and covariance structure should be specified as design choices, not selected solely to produce a preferred outcome estimate.
Choose pretreatment covariates for the propensity score and for Mahalanobis refinement; state the caliper and covariance matrix convention.
For each treated unit, retain only controls whose logit-score difference is no greater than the caliper.
Choose the eligible control with the smallest specified Mahalanobis distance, record units without partners, and assess balance and overlap.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If a control has the smallest Mahalanobis distance but fails the stated score caliper, can it be selected? Explain.
- Why does the Mahalanobis formula use the inverse covariance matrix rather than raw squared differences?
- A treated unit has no controls within its caliper. What does the matching rule require, and what design consequence should be reported?
- Does choosing the smallest Mahalanobis distance establish conditional exchangeability? Why or why not?
Solutions. (1) No. It is ineligible; distances are compared only among controls that pass the caliper. (2) The inverse covariance matrix adjusts for differing covariate scales and their correlations, so raw units do not determine the distance directly. (3) The unit remains unmatched under the stated rule; report the number and characteristics of unmatched treated units, since excluding them can change the target population. (4) No. The distance measures proximity in selected observed covariates. Conditional exchangeability concerns potential outcomes given measured covariates and cannot be established by a matching distance.