Trimming For Limited Overlap In A Propensity-Matched Study

Why Trim a Propensity-Matched Cohort?

Matching is difficult when some units have estimated propensity scores near zero or one. A treated unit with a very low estimated probability of treatment may have few comparable controls; a control with a very high estimated probability may likewise have few comparable treated units. Matching in these regions can require poor matches or rely heavily on extrapolation. Trimming removes units outside a declared score region before matching, so the analysis focuses on a population with more plausible treatment comparisons.

Trimming is not a harmless data-cleaning step. It changes who the results describe. A causal estimate after trimming generally targets an effect among the retained units, not an effect for the original cohort. The rule should therefore be defined and justified before examining outcomes, and the number and characteristics of excluded units should be reported.

Definition: Let $A_i=1$ indicate that unit $i$ received treatment and $A_i=0$ indicate control. Let $X_i$ be the measured baseline covariates and let $\hat e(X_i)$ be the estimated propensity score, the estimated probability of treatment given those covariates. A fixed-cutoff trimming rule with limits $l$ and $u$ retains unit $i$ when $l\leq\hat e(X_i)\leq u$. Write $R_i=1$ for a retained unit and $R_i=0$ otherwise.

Assumptions and the Target After Trimming

The causal interpretation still depends on the assumptions behind propensity-score adjustment. In potential-outcome notation, $Y_i(1)$ and $Y_i(0)$ are the outcomes unit $i$ would have under treatment and control. Consistency requires the observed outcome to equal the potential outcome under the treatment actually received. No interference requires one unit’s treatment not to alter another unit’s outcome. Conditional exchangeability requires $(Y(1),Y(0))$ to be independent of $A$ given measured covariates $X$; in words, the measured covariates must suffice to control confounding. Positivity requires both treatment options to have nonzero probability at covariate patterns in the target population.

For a rule that retains units according to their estimated propensity scores, the retained region is intended to avoid covariate patterns with little treatment-group overlap. If the true propensity score lies between fixed limits $l$ and $u$ within the retained region, positivity is bounded away from zero and one there. Estimated scores do not guarantee that this condition holds: a misspecified propensity model can produce misleading scores, and a nominal cutoff cannot create actual comparators where none exist. Balance and overlap must be checked again after trimming and matching.

Definition: For an ATT analysis, the target after trimming is the average treatment effect among treated units who satisfy the trimming rule: $$ \tau_{\mathrm{ATT},R}=E\left[Y(1)-Y(0)\mid A=1,\ R=1\right]. $$ This is an effect for retained treated units. It is not automatically the ATT for all treated units in the original cohort, nor the ATE for the retained cohort.

When the retention rule is a function of $X$, conditional exchangeability in the original population also holds within the retained covariate region: restricting to $R=1$ does not add information about potential outcomes once $X$ is conditioned on. But the target population has changed, and the retained group may differ from the original group in clinically important ways. A matching procedure may exclude additional units if adequate matches cannot be found; those exclusions can narrow the realized target further.

Choosing and Applying a Rule

A fixed cutoff, such as retaining estimated scores from 0.05 through 0.95, is transparent and easy to reproduce. It is only one possible rule. Another approach defines a common-support region from the observed score distributions in the two treatment groups. Such a rule should be described precisely, because “trim the non-overlap” alone does not tell the reader how units were classified. The chosen limits should be clinically and statistically defensible, specified without looking at outcomes, and accompanied by the retained score distributions.

A trimming cutoff is not the same as a matching caliper. Trimming removes all units outside a designated score region before matching. A caliper limits how far apart two units can be when a match is formed. Even after trimming, a caliper or other matching constraint may leave some units unmatched. Report these stages separately: units excluded by the trimming rule, then units omitted because no eligible match was available.

Declare the analysis target and rule.
State whether the goal is an ATT or another estimand, define the score limits or support rule, and explain why the retained region is relevant.

Count exclusions by treatment group.
Give the original and retained numbers of treated and control units, as well as exclusion percentages. A single overall percentage can hide substantial imbalance in retention by group.

Rematch and reassess diagnostics.
Re-estimate or apply the declared matching procedure in the retained cohort, then inspect score overlap and covariate balance in the matched analysis set.

Interpret the estimate for the analyzed units.
Describe who remains and state the causal assumptions. Do not generalize the effect to excluded units without additional justification.

Worked Examples

Worked Example: Counting Units Removed by a Fixed Cutoff

Suppose a hypothetical registry contains 400 treated and 600 control patients. The investigators specify, before analyzing outcomes, that they will retain units with estimated propensity scores from 0.05 through 0.95, inclusive. Among treated patients, 12 have scores below 0.05 and 28 have scores above 0.95. Among controls, 90 have scores below 0.05 and 18 have scores above 0.95. Calculate retention and exclusion by group.

For treated patients, the excluded count is $12+28=40$, so the retained count is $400-40=360$. The exclusion percentage is $40/400=0.100$, or 10.0%; retention is $360/400=0.900$, or 90.0%.

For controls, the excluded count is $90+18=108$, so the retained count is $600-108=492$. The exclusion percentage is $108/600=0.180$, or 18.0%; retention is $492/600=0.820$, or 82.0%.

Overall, $40+108=148$ of $1{,}000$ patients are removed, an overall exclusion rate of $148/1{,}000=14.8\%$. Reporting only 14.8% would obscure that control patients were excluded at a higher rate than treated patients. The post-trimming cohort has 360 treated and 492 control patients, but those counts alone do not establish that suitable matches exist. Score overlap and covariate balance still need to be examined.

Worked Example: Rechecking Balance and Overlap

In a hypothetical matched cohort, age has a mean of 62 years with standard deviation 10 years among treated patients and a mean of 55 years with standard deviation 12 years among controls before trimming. After applying the declared cutoff and rematching, the corresponding means are 60 and 59 years, with standard deviations 9 and 10 years. Calculate the standardized mean difference for age before and after the procedure.

For a continuous covariate, use the difference in means divided by the pooled standard deviation: $$ \mathrm{SMD}=\frac{\bar{x}_1-\bar{x}_0}{\sqrt{(s_1^2+s_0^2)/2}}. $$ Before trimming and matching, the pooled standard deviation is $\sqrt{(10^2+12^2)/2}=\sqrt{122}=11.05$ years. Thus $(62-55)/11.05=0.634$. The absolute standardized difference is 0.634, indicating substantial age imbalance before the procedure.

After trimming and rematching, the pooled standard deviation is $\sqrt{(9^2+10^2)/2}=\sqrt{90.5}=9.513$ years. The standardized difference is $(60-59)/9.513=0.105$. Age balance has improved considerably, but an absolute difference of 0.105 is still just above the commonly used 0.10 diagnostic benchmark. That benchmark is a heuristic, not a formal test that proves balance.

Now examine the retained estimated-score distributions by bins. Suppose the treated and control counts in the bins 0.05–0.25, 0.25–0.50, 0.50–0.75, and 0.75–0.95 are respectively 45 and 150, 105 and 170, 120 and 120, and 90 and 52. Both groups occur in each bin, which is evidence of broad overlap at this resolution. It does not prove unit-level overlap or adequate balance: within-bin scores can still differ, and age is not the only covariate to check. The next diagnostic step is to inspect score distributions and all prespecified baseline covariates in the matched sample, not merely the retained pre-match cohort.

Worked Example: Interpreting the Effect for the Retained Treated Group

In another hypothetical study, trimming leaves 160 treated patients and 220 controls. A matching procedure pairs each retained treated patient with a control, allowing control reuse. The mean of the 160 within-pair outcome differences is 1.30 points, where a positive difference means a higher outcome under treatment. An appropriate variance procedure that accounts for the matching design gives a standard error of 0.459 points. Calculate a normal-approximation 95% confidence interval and state the target.

The matched estimate is the average of the treated-minus-control outcome differences: $$ \hat{\tau}_{\mathrm{ATT},R}=\frac{1}{160}\sum_{i=1}^{160}(Y_{i,\mathrm{treated}}-Y_{i,\mathrm{matched\ control}})=1.30\ \text{points}. $$ The 95% interval is the estimate plus or minus 1.96 standard errors: $$ 1.30\pm1.96(0.459)=1.30\pm0.8996, $$ which gives $[0.400,\ 2.200]$ points after rounding.

Under the stated causal assumptions and the validity of the matching and variance procedures, the estimated treatment effect is an increase of 1.30 points among the retained treated patients. The confidence interval gives a range of values compatible with the data and method at the 95% confidence level; it does not mean there is a 95% probability that the fixed effect lies in this particular interval. The result does not directly estimate the effect among treated patients removed by trimming, nor among all patients in the original cohort.

Why the Retained Population Matters

The derivation of the changed target is direct. Let $R$ indicate membership in the retained region. The original treated-group effect averages $Y(1)-Y(0)$ over all treated units. After trimming, the same difference is averaged only over treated units with $R=1$: $$ E[Y(1)-Y(0)\mid A=1,R=1]. $$ Unless the effect is identical in retained and excluded treated units, this conditional mean need not equal $E[Y(1)-Y(0)\mid A=1]$. Thus an estimate can change because trimming improves comparability, because it changes the composition of the target population, or both. A change in the estimate alone cannot establish which explanation is responsible.

Diagnostics should be reported after the full design process. Examine the propensity-score distributions by treatment group before and after trimming; report group-specific retained counts and score ranges; and assess standardized differences for every important baseline covariate in the matched sample. Distributional checks can reveal problems that means alone miss. If scores still show little overlap, if important covariates remain imbalanced, or if many treated units cannot be matched, the planned design has not achieved its goal. Revisit the model or the target population, and report any revised analysis transparently.

Trimming can reduce reliance on extreme comparisons, but it cannot repair unmeasured confounding, incorrect covariate measurement, or a misspecified propensity model. Nor does trimming guarantee adequate overlap: the estimated score can conceal sparse regions or omit relevant covariate dimensions. If the retained target is too narrow to answer the clinical question, the limitation is substantive, not merely technical. Report who was removed and consider whether the narrower target remains useful.

Common Pitfall and Alternatives

A common mistake is to write, “We removed extreme propensity scores, so the groups are now comparable and the original-cohort effect is unbiased.” This reasoning is wrong twice. A cutoff does not demonstrate balance or eliminate unmeasured confounding, and discarding units changes the target population. The correction is to treat trimming as a design restriction: state the rule, show the group-specific exclusions, rematch, reassess diagnostics, and interpret the result for the retained population under explicit assumptions.

The main alternative is to retain the full cohort and use a matching or adjustment strategy that can represent the original target, if adequate comparators exist. That avoids the specific target restriction caused by trimming but may leave estimates dependent on weak comparisons or extrapolation. A caliper can prevent distant matches but may itself leave units unmatched; that is a separate source of sample restriction and should be reported. No approach is automatically preferable: the choice depends on the scientific target, actual overlap, and the consequences of excluding units. Sensitivity analyses using defensible alternative cutoffs can show whether conclusions depend strongly on the chosen restriction, but they should not be used to select the most favorable result.

Key takeaway: Trimming defines a more restricted analysis population; it does not certify causal validity. Report the rule and exclusions, verify overlap and balance after rematching, and name the retained population when interpreting the effect.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A cohort has 250 treated and 500 control units. A score cutoff excludes 25 treated and 100 controls. What are the retained counts and exclusion percentages by group?
Why is a propensity-score trimming cutoff not the same as a matching caliper?
What population does a trimmed ATT estimate target, and why might it differ from the ATT in the original treated group?
After trimming, both groups appear in every broad propensity-score bin, but one important baseline covariate has an absolute standardized mean difference of 0.24. What should the analyst conclude or do?

Solutions. (1) Retained treated units: $250-25=225$; exclusion percentage: $25/250=10.0\%$. Retained controls: $500-100=400$; exclusion percentage: $100/500=20.0\%$. (2) Trimming removes units outside the declared score region before matching; a caliper limits the distance allowed between matched units and can leave units unmatched even inside the retained region. (3) It targets the average treatment effect among treated units who satisfy the trimming rule, $E[Y(1)-Y(0)\mid A=1,R=1]$. It can differ because excluded treated units may have different treatment effects and because the averaging population is narrower. (4) Broad bin overlap does not establish covariate balance. The analyst should inspect the covariate’s distribution, reconsider the propensity model or matching design, and report the remaining imbalance; a causal interpretation should not claim that balance has been achieved.