Sensitivity of the Target to Matching Ratio

Why the Matching Ratio Can Matter

Matching one control to each treated unit and matching several controls to each treated unit are not simply two ways to obtain different sample sizes. The ratio can change which treated units are retained, how much influence each retained unit receives, and which controls represent them. Those changes may alter the population to which an estimate refers. They may also change an estimate while leaving its target population unchanged, for example because a different control group leaves different residual imbalance.

The distinction matters when an analyst reports that a treatment-effect estimate is “sensitive to the matching ratio.” That statement alone does not tell us why the estimate changed. We must compare the matched treated populations and their weights, assess effect heterogeneity, and examine how well each ratio produces comparable treated and control groups. Matching is a design choice; it does not by itself guarantee that a contrast has a causal interpretation.

Definition: For matching ratio \(r\), let \(I_r\) be the treated units represented in the matched analysis and let \(a_{ir}\) be their normalized target weights, with \(a_{ir}\geq 0\) and \(\sum_{i\in I_r}a_{ir}=1\). The ratio-specific empirical target population assigns weight \(a_{ir}\) to treated unit \(i\). If each retained treated unit receives equal weight, \(a_{ir}=1/|I_r|\). If a procedure uses different weights, those weights—not merely the number of controls per treated unit—define the target.

Write \(Y_i(1)\) and \(Y_i(0)\) for unit \(i\)'s potential outcomes under treatment and control, and define the unit treatment effect as \(\tau_i=Y_i(1)-Y_i(0)\). For a target with weights \(a_{ir}\), the corresponding empirical average effect is

\tau_r=\sum_{i\in I_r}a_{ir}\tau_i.

This expression makes the key point explicit: a ratio changes the causal target when it changes the represented units or their relative weights and treatment effects vary across those units. If all treated units remain in the analysis and each receives equal weight at every ratio, the treated-unit target is unchanged. The ratio can still change the estimate, because the controls and their weights differ.

Conditions for a Causal Interpretation

The target comparison is meaningful only alongside the conditions needed for the matched contrast to estimate it. Potential outcomes must be well-defined and consistent with observed treatment and outcome. Conditional exchangeability requires that, given the pretreatment covariates used to justify the design, treatment assignment carries no additional information about the potential outcomes. Positivity requires a positive chance of both treatment assignments in the covariate regions represented by the target. Matching must also yield adequate covariate comparability within the matched analysis.

These conditions are not established by choosing a larger ratio. More controls can improve precision when they are sufficiently comparable, but including less comparable controls can worsen balance. Conversely, a restrictive design may exclude treated units and thereby change the target. Any causal claim is conditional on the design assumptions being credible for the particular ratio and target.

The earlier material on matching weights and induced populations provides the general principle: weights determine which covariate distribution an estimate summarizes. For a ratio-sensitivity analysis, apply that principle separately to every ratio. Do not infer the target from the label “1:1” or “1:3” alone; inspect which units entered and what weights they received.

A Decomposition of Ratio Differences

Let \(\widehat{\tau}_r\) be the effect estimate under ratio \(r\). Add and subtract the target effect \(\tau_r\) to compare two ratios:

\widehat{\tau}_{r_2}-\widehat{\tau}_{r_1} = (\tau_{r_2}-\tau_{r_1}) + \{(\widehat{\tau}_{r_2}-\tau_{r_2})-(\widehat{\tau}_{r_1}-\tau_{r_1})\}.

The first term is the difference in target effects. The second is the difference in estimation errors, which can include sampling variation and bias from residual imbalance or other failures of the design assumptions. This identity is exact; it does not make the two components observable. In practice, covariate distributions and effect-modifier patterns help assess whether a composition change is plausible, while balance diagnostics and uncertainty assessments help evaluate estimation differences.

For a common covariate partition with categories \(x\), suppose the effect is constant within each category, with value \(\tau(x)\), and ratio \(r\) gives category weight \(q_r(x)\). Then

\tau_{r_2}-\tau_{r_1} = \sum_x\{q_{r_2}(x)-q_{r_1}(x)\}\tau(x).

A change in covariate composition changes the target effect only to the extent that effects differ across the categories whose weights changed. If effects are constant across all categories, a composition shift alone does not change the average effect, although it still changes the population described by the estimate.

Worked Example: A Ratio Changes Which Treated Units Are Represented

Suppose a hypothetical analysis has two pretreatment risk groups, lower-risk and higher-risk. A 1:1 design retains four lower-risk and two higher-risk treated units. A 3:1 design retains three lower-risk and one higher-risk treated unit because the higher ratio, under the chosen no-replacement and eligibility rules, cannot form acceptable matched sets for the other treated units. Assume, for illustration, that effects are 2 outcome points in the lower-risk group and 8 points in the higher-risk group, and that each retained treated unit receives equal weight.

Under 1:1 matching, the higher-risk share is \(2/6\), so the target effect is

\tau_1=\frac{4}{6}(2)+\frac{2}{6}(8) =\frac{8+16}{6}=4.000\text{ points}.

Under 3:1 matching, the higher-risk share is \(1/4\), giving

\tau_3=\frac{3}{4}(2)+\frac{1}{4}(8) =1.5+2=3.500\text{ points}.

The target effect is 0.500 points lower under 3:1 matching. Under the example's assumptions, this difference is entirely attributable to the different composition of the retained treated units: the higher-effect group has less weight under 3:1. In an actual analysis, these values would not be known directly; the example separates the target-population mechanism from the additional uncertainty of estimating effects.

When the Target Stays Fixed but the Estimate Changes

If the same treated units remain and receive the same target weights at each ratio, then \(\tau_{r_2}-\tau_{r_1}=0\). A difference between estimates cannot then be attributed to a change in that treated target. It may arise from sampling variation, different control weights, or residual covariate imbalance. If the controls differ on prognostic variables, even an unchanged treated composition does not ensure equally credible estimates.

Worked Example: Same Target, Different Matched Contrasts

Consider a hypothetical study with two lower-risk and two higher-risk treated units, all retained with equal weight under both ratios. Their observed treated-outcome means are 12 points for the lower-risk group and 28 points for the higher-risk group. With 1:1 matching, the corresponding matched control-outcome means are 10 and 20 points. The group contrasts are \(12-10=2\) and \(28-20=8\), so the equally weighted estimate is

\widehat{\tau}_1=\frac{2+2+8+8}{4}=5.000\text{ points}.

With 3:1 matching, suppose the matched controls' outcome means are 9 and 20 points for the two groups. The group contrasts are now \(12-9=3\) and \(28-20=8\), giving

\widehat{\tau}_3=\frac{3+3+8+8}{4}=5.500\text{ points}.

The treated target remains two units in each risk group, so its composition has not changed. The 0.500-point estimate difference instead reflects a changed control contrast in the lower-risk group. For example, the additional controls might differ on an unmeasured prognostic feature, or the observed means might vary by chance. The changed estimate is not evidence that the target population changed. Nor do the arithmetic calculations establish that either estimate is unbiased.

Diagnostics and Practical Comparisons

For each ratio, report the number and proportion of treated units retained, the distribution of the relevant covariates among those units, and the weights that define their contribution to the estimate. Compare matched covariate distributions across ratios, not only treated-versus-control balance within each ratio. If treatment effects may vary across measured covariates, examine whether the ratio changes the representation of those covariates. A shift in a potential effect modifier is especially relevant to interpreting different target effects.

Also assess treated-control balance separately at each ratio. Useful checks include covariate mean differences, distributional comparisons, and overlap in the covariate regions used for matching. Good balance within each ratio does not imply that both ratios target the same population. Likewise, similar treated covariate distributions do not prove that the estimates differ only by chance: control-side balance and unmeasured factors still matter.

A practical comparison can be organized as follows:

Question	Evidence to inspect	Interpretation
Did the treated target change?	Retained treated units and their normalized weights	Different units or weights imply different empirical target distributions.
Could composition explain an effect change?	Changes in covariate distributions, especially measured effect modifiers	Composition is a plausible explanation when effect heterogeneity aligns with the changed weights.
Did the matched contrast change in quality?	Treated-control balance and overlap at each ratio	New imbalance can change an estimate without changing the treated target.
How stable are the estimates?	Uncertainty estimates and sensitivity across reasonable designs	Instability warrants explanation; it does not identify which design is correct.

The main alternatives answer different questions. A fixed treated-unit analysis can keep the target constant while varying the number of controls, provided all treated units remain and retain the same weights. A ratio-specific matched analysis may instead allow the retained treated set to change, and therefore may change the target. Standardizing ratio-specific effects to one chosen covariate distribution can help compare effects on a common target, but requires defensible estimates of effects across that distribution. No alternative removes the need to justify comparability and overlap.

Worked Example: Separating Composition Change From Estimate Change

Suppose a hypothetical 1:1 analysis gives treated-target weights of 0.80 for a lower-effect group and 0.20 for a higher-effect group. Under 3:1 matching, the weights are 0.40 and 0.60. Assume the group effects are 2 and 8 points. The 1:1 target effect is

\tau_1=0.80(2)+0.20(8)=1.6+1.6=3.200\text{ points},

and the 3:1 target effect is

\tau_3=0.40(2)+0.60(8)=0.8+4.8=5.600\text{ points}.

The target composition component is \(5.600-3.200=2.400\) points. Now suppose the estimated effects are 3.4 points and 5.0 points, respectively. Their observed difference is \(1.600\) points, not 2.400. The decomposition shows that the difference in estimation errors is

(5.000-5.600)-(3.400-3.200) =-0.600-0.200=-0.800\text{ points}.

Thus, for these stipulated numbers, the estimated difference combines a 2.400-point target shift with a \(-0.800\)-point difference in estimation errors. In real data, the true group effects are generally unknown, so this exact decomposition cannot usually be calculated. The example shows why a single estimate difference should not automatically be attributed to changing population composition.

Common Pitfalls and Limitations

Assuming more controls means a different target. If the same treated units retain equal weights, adding controls does not itself change the treated-unit target. It changes the control construction and may affect precision or bias.
Assuming the target is fixed because the treated count is fixed. The same number of treated units can have different identities or weights. Compare their covariate distributions and weights, not just the counts.
Calling every estimate change a population shift. If the target weights are unchanged, investigate control imbalance, sampling variation, and other sources of estimation error.
Calling good balance proof of causality. Balance on measured covariates does not establish exchangeability with respect to unmeasured causes, and limited overlap can restrict the target.
Choosing a ratio only because its estimate is preferred. Report the design rationale and target implications. Selecting among designs based on the resulting effect can make the reported estimate difficult to interpret.

Ratio sensitivity is therefore a diagnostic and interpretive exercise, not a rule that one ratio is universally best. Larger ratios may use more controls, but can require less comparable matches or alter which treated units are represented. Smaller ratios may preserve a more restricted matched sample, yet use fewer controls. The appropriate choice depends on the target question, overlap, matching quality, and the design's assumptions. Report the target alongside the estimate so readers can distinguish a change in the effect being summarized from a change in how that effect was estimated.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If two matching ratios retain precisely the same treated units with identical weights, has the treated target changed? What could still cause the estimates to differ?
A ratio change alters the target weight on a group from 0.25 to 0.50. The group effect is 6 points, while the other group's effect is 2 points. What is the target-effect change due solely to this reweighting?
Why is the number of retained treated units alone insufficient to determine whether two ratios target the same population?
What does a within-ratio balance check establish, and what does it not establish about comparisons across ratios?

Solutions. (1) No; the represented treated units and their weights are unchanged. Different controls, residual imbalance, or sampling variation can still change the estimate. (2) The weight shift is 0.25 toward a group whose effect exceeds the other group's by \(6-2=4\) points. The target effect changes by \(0.25(4)=1.000\) point. (3) Equal counts can represent different treated units or different weights, which produce different covariate distributions and potentially different target effects. (4) It assesses treated-control comparability on measured covariates within that ratio. It does not show that the treated targets are the same across ratios or establish exchangeability on unmeasured factors.