From a Registry Question to an ATE-Oriented Match
A health registry can contain detailed treatment and outcome records without random treatment assignment. For example, patients who receive a new care program may differ from untreated patients in age, disease severity, or prior health-care use. Matching can make the comparison more credible by pairing patients with similar measured pretreatment characteristics. It cannot, by itself, remove confounding by characteristics that were not measured or included in the analysis.
The target must be specified before interpreting a matched contrast. Here the target is the average treatment effect (ATE) for a defined registry population: the average difference between the outcome each person would have under treatment and the outcome that same person would have without treatment. Matching treated patients to controls while retaining only the treated patients is generally ATT-oriented. Equal numbers of treated and control patients do not, on their own, change that target to an ATE.
Let \(X_i\) denote pretreatment covariates, and define the propensity score as \(e(X_i)=P(Z_i=1\mid X_i)\). In practice, it is commonly estimated with a model such as logistic regression. Similar propensity scores can indicate similarity in measured covariates when the score model is adequately specified, but balance must still be checked directly.
A causal interpretation for the registry ATE requires, at minimum, consistency (the observed outcome equals the potential outcome under the treatment received), no interference between patients, conditional exchangeability \((Y(1),Y(0))\perp Z\mid X\), and positivity \(0<P(Z=1\mid X)<1\) throughout the target population. The propensity-score model must also be adequate for the matching design, and the resulting matches must represent the target population well. These assumptions are not established merely by producing a matched data set.
Estimating Scores and Forming Matches
The following small calculation illustrates how a fitted logistic model produces a score. Suppose a hypothetical registry model is \(\operatorname{logit}\{e(X)\}=-2+0.04(\text{age}-60)+0.70D\), where \(D=1\) indicates diabetes. The score is the predicted probability of receiving treatment given those covariates; it is not a predicted outcome or a treatment effect.
Worked Example: Calculating a Propensity Score
Hypothetical patient. A 62-year-old patient with diabetes has linear predictor \(\eta=-2+0.04(62-60)+0.70(1)=-2+0.08+0.70=-1.22\).
Convert the log odds to a probability:
The model assigns this patient an estimated 22.79% probability of treatment, conditional on the variables in the model. A 59-year-old patient without diabetes would have \(\eta=-2+0.04(59-60)=-2.04\), giving \(\hat e(X)=\exp(-2.04)/[1+\exp(-2.04)]\approx 0.1151\). These are model-based estimates, not known treatment probabilities.
For an ATE-oriented matching estimator, each patient in the target sample needs a comparison from the opposite treatment group. A simple design in a balanced sample is to make one-to-one matches without replacement, using nearest propensity scores, and retain all patients. In an unbalanced registry, a design must still account for the full target population; depending on the data, that can require reuse of comparison patients, multiple matches, or a different matching or weighting design. A treated-only match does not become an ATE design simply by matching every treated patient.
The matching formula is only a useful ATE estimator when the matched comparisons support the target: the covariate distribution represented by the matched patients must be close to that of the specified registry population. Poor overlap, discarded patients, or substantial residual imbalance may make the registry-wide ATE unsupported. In those circumstances, report a narrower matched-population target or choose a different design; do not label a selected-sample contrast as the registry ATE.
Worked Registry Analysis
Consider a hypothetical registry of patients offered a care program, with systolic blood pressure measured 12 weeks later. The continuous outcome is measured in mm Hg. A logistic model using pretreatment age, diabetes, and prior utilization produces the propensity scores shown below. The analyst makes four one-to-one nearest-score matches without replacement. In this deliberately small example, all eight patients are retained, and each match has a score difference of 0.01.
| Pair | Treated score | Control score | Treated outcome (mm Hg) | Control outcome (mm Hg) | Treated minus control (mm Hg) |
|---|---|---|---|---|---|
| A | 0.20 | 0.21 | 144 | 148 | -4 |
| B | 0.31 | 0.30 | 138 | 144 | -6 |
| C | 0.46 | 0.45 | 150 | 152 | -2 |
| D | 0.59 | 0.60 | 146 | 154 | -8 |
Worked Example: Balance, ATE Estimate, and Conditional Uncertainty
Check measured balance. Suppose the corresponding ages are 50, 60, 52, and 61 years for treated patients and 51, 59, 50, and 62 years for controls. Their means are \(55.75\) and \(55.50\) years. The pooled within-group standard deviation is \(s_p=\sqrt{(92.75+105)/6}=\sqrt{32.9583}=5.741\) years, so the standardized mean difference is \((55.75-55.50)/5.741=0.04355\). The diabetes proportions are 0.50 in both groups, giving a standardized difference of 0. The table's match construction retains all patients, so these balance summaries also describe the matched groups. This is reassuring for these two covariates, but balance should be checked for every important pretreatment covariate, not inferred from close scores alone.
Estimate the effect. The pair differences are \(-4,-6,-2,-8\) mm Hg. Because every patient appears once and the treated and control group sizes are equal, the two-sided imputation estimator gives each pair difference twice in the eight-patient total:
Equivalently, the mean of the four matched-pair differences is \((-4-6-2-8)/4=-5.00\) mm Hg. The registry comparison estimates a 5.00 mm Hg lower 12-week systolic blood pressure under treatment than under no treatment, for the population represented by these matched patients, subject to the causal assumptions and the adequacy of matching.
Describe conditional uncertainty. For these four pair differences, the sample standard deviation is \(s_d=\sqrt{[(1)^2+(-1)^2+(3)^2+(-3)^2]/3}=\sqrt{20/3}=2.582\) mm Hg. Thus \(SE(\bar d)=s_d/\sqrt{4}=1.291\) mm Hg. If pair differences are independent draws from a normal distribution, a paired-\(t\) interval with \(df=3\) is \(\bar d\pm t_{0.975,3}SE=-5.00\pm3.182(1.291)=-5.00\pm4.108\), or \((-9.108,-0.892)\) mm Hg. Under those conditions, the interval procedure has 95% repeated-sampling coverage for its target. This tiny example makes the normality assumption strong; moreover, this simple paired interval conditions on the constructed matches and does not fully account for estimating the propensity model or selecting matches. A registry analysis should choose an uncertainty method suited to its matching estimator and sampling design.
The balance calculation illustrates why propensity-score proximity is not itself a diagnostic endpoint. A score can be similar while particular covariates remain imbalanced, especially if the model is misspecified or important variables are omitted. Review standardized differences and covariate distributions before and after matching, and examine whether patients in the target population have plausible counterparts across treatment groups. The earlier material on covariate balance diagnostics and overlap reporting gives the relevant reporting framework.
Why the Target Population Matters
Worked Example: Why Treated-Only Matching Does Not Generally Estimate the ATE
Suppose a hypothetical registry target population is 80% high-risk and 20% low-risk. Assume, for illustration, that the true treatment effect is 10 outcome units for high-risk patients and 2 units for low-risk patients. Its ATE is \(0.80(10)+0.20(2)=8.4\) units.
Now suppose only 20% of the treated group is high-risk and 80% is low-risk. A design that matches every treated patient to a control and averages only those treated patients' contrasts targets their covariate distribution. Its effect is \(0.20(10)+0.80(2)=3.6\) units, the ATT for this example, not the target population's ATE of 8.4 units. Matching may make the treated and control groups comparable within the treated distribution, but it does not change which population receives the averaging weights.
Matching is one approach among several. Standardization models outcomes conditional on covariates and averages predicted outcomes over the target population; weighting reweights observations to represent a specified target. Each approach depends on adequate confounding control and overlap, and each can be unreliable when those conditions fail. Matching is especially useful when a transparent set of patient-to-patient comparisons is valuable, but it can discard data, depend on matching choices, and leave residual imbalance. The effect scale also matters: the example reports a mean difference in mm Hg, not a relative or percentage change.
Diagnostics, Limitations, and Common Mistakes
State whether the estimand is the registry ATE, an ATT, or an effect for a restricted matched population. Report any exclusions or patients without matches.
Check score distributions by treatment group and the distances for individual matches. A forced distant match is not made credible by its label as a match.
Compare important pretreatment variables before and after matching using balance summaries and distributional checks. Revisit the model or design if important differences remain.
Give the outcome scale and units, the target population, the matched sample, and an interval method whose assumptions correspond to the matching design.
- Mistake: “There are equal numbers in each group, so this is an ATE.” Equal sample sizes do not determine the target. If every treated patient is retained and controls are selected to resemble them, the comparison is ordinarily ATT-oriented. To claim a registry ATE, the design and averaging must represent the registry target population.
- Mistake: treating a high propensity score as a high expected outcome. The score predicts treatment conditional on measured covariates; it does not predict the clinical outcome or the benefit of treatment.
- Mistake: treating good measured balance as proof of no confounding. Balance diagnostics assess recorded covariates. They cannot establish conditional exchangeability with respect to unmeasured factors.
- Mistake: applying a routine paired interval without describing its scope. The interval may describe variation in matched differences under its assumptions, while omitting uncertainty from score estimation, match selection, or a restricted target. State what the calculation covers.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A registry has 200 treated patients and 800 controls. An analyst matches each treated patient to a control and averages the 200 differences. What target does this design ordinarily represent?
- A matched-pair analysis has pair differences of \(-3\), \(-5\), and \(-4\) mm Hg. What is the mean difference, and how should its sign be interpreted?
- Why does a propensity-score difference of nearly zero not guarantee covariate balance?
- Name two assumptions needed for a causal interpretation of a registry ATE and explain their roles.
Solutions. (1) Ordinarily the ATT, or an effect for the matched treated population, because the averaging is over treated patients. (2) The mean is \((-3-5-4)/3=-4.00\) mm Hg; treatment is associated with an estimated 4.00 mm Hg lower outcome in the matched comparison, with causal interpretation conditional on design assumptions. (3) The score model may be misspecified, important covariates may be omitted, or particular variables may remain imbalanced despite similar scores; balance must be assessed directly. (4) Conditional exchangeability requires that measured covariates suffice to remove treatment-outcome confounding; positivity requires both treatment options to be possible across the target covariate patterns. Consistency and no interference are also needed to link observed outcomes to well-defined potential outcomes.