Full Matching For A Comparative Effectiveness Study

Why Full Matching Can Help

A comparative-effectiveness cohort may contain many treated individuals but relatively few comparable controls in some parts of the covariate distribution, and the reverse in other parts. A fixed 1:1 match can discard usable observations or force a uniform ratio where the data do not support one. Full matching instead groups retained treated and control individuals into matched sets with variable treatment-to-control ratios. For an average treatment effect on the treated (ATT), the sets can be weighted so that each treated individual contributes equally to the treated outcome mean and the controls collectively represent the treated group.

The method does not make treatment assignment random. Its causal interpretation still depends on measured covariates adequately controlling confounding, adequate overlap, and a correctly defined treatment, outcome, and follow-up period. The introductory material on propensity score matching provides the broader context; here the emphasis is on the mechanics and checks in a worked comparative-effectiveness study.

The weights make the control group’s weighted size equal the treated group’s size, both overall and within each matched set. If the retained treated group has \(n_T=\sum_j n_{Tj}\) individuals, the weighted ATT contrast is the treated mean outcome minus the weighted control mean outcome. It targets the ATT for the treated individuals retained in the analysis, not automatically for every treated individual in the original cohort.

Equivalently, define the within-set contrast \(\widehat{\Delta}_j=\bar{Y}_{Tj}-\bar{Y}_{Cj}\). Then \(\widehat{\mathrm{ATT}}=\sum_j(n_{Tj}/n_T)\widehat{\Delta}_j\). Sets contribute in proportion to their number of treated individuals: this is important when set sizes differ.

Assumptions Before Interpreting the Contrast

The matching calculation is descriptive until causal identification assumptions are made. Let \(Y_i(1)\) and \(Y_i(0)\) be individual \(i\)’s potential outcomes under treatment and control. The ATT is \(E[Y(1)-Y(0)\mid A=1]\) in a specified target population. A causal interpretation for the matched estimate requires, at minimum, the following conditions:

• Consistency and a well-defined treatment: the observed outcome equals the potential outcome for the treatment actually received, and treatment versions and follow-up are defined clearly.
• Conditional exchangeability: given the measured baseline covariates \(X\), treatment is independent of the potential outcomes. In particular, there are no unmeasured confounders remaining after conditioning on \(X\).
• Positivity for the treated target: treated individuals have a nonzero probability of receiving control at their covariate values, so suitable controls exist across the covariate profiles represented by the target treated group.
• No interference: one individual’s treatment does not change another individual’s outcome, unless the estimand and design explicitly account for such interference.

Full matching may use an estimated propensity score \(e(X)=P(A=1\mid X)\), a distance based on covariates, or a specified combination. A good-looking match cannot establish exchangeability, and weighting cannot manufacture control information where there is no overlap. The target population must also be reported accurately if some treated individuals are excluded for lack of comparable controls.

Outcome Contrast and a Justification for the Weights

Consider hospital length of stay, measured in days, as a hypothetical outcome. Suppose the treated outcomes in sets A through D are respectively \(10,12\); \(8,10\); \(7,9\); and \(11,13\). Suppose the control outcomes are respectively \(12,14\); \(10,11,12\); \(9,11\); and \(13,15\). Each set’s treated mean is two days below its control mean.

The weighting logic follows directly from the set structure. In set \(j\), the weighted control mean is \(\bar{Y}_{Cj}\), because multiplying every control by \(n_{Tj}/n_{Cj}\) changes the total weight but not the mean. The set’s weighted control total is \(n_{Tj}\bar{Y}_{Cj}\). Summing across sets and dividing by \(n_T\) gives a control mean in which each set is represented in proportion to its treated count. Subtracting that mean from the treated mean gives the stated stratified ATT contrast.

Assessing Balance Before Reading the Outcome Result

Matching should be evaluated using baseline covariates, not by asking whether treatment and control groups have statistically significant differences in those covariates. Standardized mean differences (SMDs), weighted means and proportions, and distributional summaries describe the magnitude of imbalance without making balance depend on sample size. Examine covariates used in the design and important prognostic variables, including nonlinear terms or interactions when scientifically relevant. Similar means alone can hide differences in spread or distribution.

For a continuous covariate, one common diagnostic is the difference between the treated mean and weighted control mean divided by a prespecified reference standard deviation, often the pooled standard deviation before matching. For a binary covariate, the difference in proportions can be divided by the standard deviation based on the average of the two proportions. The chosen convention should be stated and used consistently; an SMD is a diagnostic, not a hypothesis test or a universal pass/fail guarantee.

Balance diagnostics should be computed after applying the actual analysis weights. Also inspect propensity-score overlap and the distribution of weights. A full-matching set with many treated individuals but very few controls gives each control a larger weight; such a set may reveal limited local overlap and can make the estimate sensitive to a small number of controls. Report set sizes, exclusions, weight ranges, and important weighted balance results so readers can judge what population and comparisons support the estimate.

Uncertainty for a Matched-Set Contrast

One model-based variance calculation treats matched sets as independent and estimates uncertainty within each set. Write \(s^2_{Tj}\) and \(s^2_{Cj}\) for the sample variances of the treated and control outcomes in set \(j\). Under independent sampling within treatment groups and sets, and conditional on the matched-set structure, the estimated variance of the set contrast is \(s^2_{Tj}/n_{Tj}+s^2_{Cj}/n_{Cj}\). This sample-variance estimator requires \(n_{Tj}\ge 2\) and \(n_{Cj}\ge 2\) in every set; if either group is a singleton, this formula is undefined for that group, and an alternative variance model or design-appropriate method is needed. Since the overall ATT weights set \(j\) by \(n_{Tj}/n_T\), independence across sets gives the following estimated variance:

This is a conditional, model-based calculation, not a universal variance formula for every matching procedure. In particular, it does not automatically account for uncertainty from estimating the propensity score or selecting the matched sets. A design-appropriate robust or resampling procedure may be preferable, and resampling should repeat the relevant design steps when the aim is to include their uncertainty.

Comparisons, Failure Modes, and Limits

Full matching is useful when the data support different local treated-to-control ratios and the analyst wants to retain a broad set of comparable observations. Fixed-ratio matching instead imposes the same number of controls per treated individual and may discard otherwise useful observations. Direct outcome regression can adjust for measured covariates without forming matched sets, but depends on its outcome-model specification. Weighting without matching is another alternative, but can be especially sensitive to extreme weights. None of these approaches resolves unmeasured confounding merely by changing the calculation.

Common warning signs include poor weighted balance, treated individuals without comparable controls, a few controls carrying a large share of the total weight, and a small number of matched sets driving the outcome contrast. If matching discards treated individuals, the estimand changes to the ATT among those retained unless a broader target is justified. If the outcome is missing differentially or follow-up differs between groups, the simple weighted contrast may not answer the intended comparative-effectiveness question.

• Mistake: “The matching algorithm formed sets, so baseline confounding is controlled.” Correction: Matching only uses the variables and distance structure specified; inspect weighted balance and explain why measured covariates are adequate, while acknowledging possible unmeasured confounding.
• Mistake: “Every control should have weight 1 because every person is one observation.” Correction: For this ATT construction, each control in set \(j\) has weight \(n_{Tj}/n_{Cj}\), ensuring the set’s total control weight equals its treated count.
• Mistake: “A small p-value for a baseline difference proves the match failed,” or “a large p-value proves balance.” Correction: Use weighted effect-size and distributional diagnostics; balance tests are strongly affected by sample size.