How Trimming Changes the Target Population

Trimming Defines Who the Analysis Describes

A propensity-score restriction does more than remove observations that may be difficult to compare. It also changes the population represented by an analysis. If units are retained according to their propensity scores, covariate profiles with scores inside the specified region become more prominent in the retained group, while profiles outside it disappear. Consequently, a treatment effect for the retained group need not equal the effect for the original population.

This tutorial focuses on that change in target population. It treats the score region as specified in advance and asks: what population remains, how does its covariate composition differ, and what does an average treatment effect in that population mean? It does not assume that trimming itself establishes causal identification.

For an interval \(\mathcal{S}=[l,u]\), retention means \(l\le e(X)\le u\). The region may be described more generally as a set of score values. The definition uses the population propensity score to make the target population precise; in practice, the score is usually estimated. The consequences of estimating it require additional care because the retained group can then depend on the fitted model and the observed sample.

The change in population can be described without making any causal assumptions. Causal interpretation requires more. Let \(Y(1)\) and \(Y(0)\) be the potential outcomes under treatment and control. If consistency holds, treatment is conditionally exchangeable given \(X\), and \(0<e(X)<1\) for units retained, then an average treatment effect for the retained population is well-defined and identified under those assumptions. Here conditional exchangeability means \((Y(1),Y(0))\perp A\mid X\); consistency means the observed outcome equals the potential outcome under the treatment actually received. These conditions support causal interpretation, but they do not imply that the retained and original populations have the same covariate distribution or the same average treatment effect.

How the Covariate Distribution Changes

Write \(P(R=1)\) for the fraction of the original population retained. For any covariate profile \(x\), the retained distribution is the original distribution restricted to profiles whose scores fall in the region, then normalized by the retained fraction. If \(X\) has a density, this is expressed as follows; for discrete covariates, the same relationship applies to probability masses.

The numerator removes the density of excluded profiles. Dividing by \(P(R=1)\) makes the retained density integrate to 1. Thus, for any function \(h(X)\) whose expectation exists, the retained-population expectation is

This formula gives a direct way to calculate composition changes. For a binary covariate, set \(h(X)\) equal to its indicator; the resulting expectation is its prevalence. For a quantitative covariate, set \(h(X)=X\); the result is its retained-population mean. The formula also shows why the retained fraction alone does not describe the change: two restrictions can retain the same fraction but retain quite different covariate profiles.

The corresponding causal target, when the assumptions stated above hold, is

This is generally different from the original-population average treatment effect \(E\{Y(1)-Y(0)\}\). They are equal in special cases, such as when the conditional average treatment effect does not vary across the profiles affected by the restriction. Equality is not guaranteed merely because the same outcome and treatment are studied.

Worked Changes in Means and Treatment Effects

A retained mean is calculated by dividing the covariate total among retained units by the retained population fraction. Equivalently, it is a weighted average of the group means using the groups’ shares within the retained population. The following example illustrates why the numerator and denominator must both correspond to the restriction.

A change in covariate composition matters for an average treatment effect when effects vary across covariate profiles. To see this, suppose the population consists of groups with different conditional average treatment effects. The restricted effect weights group effects by their shares among retained units, not by their shares in the full population.

Assumptions, Diagnostics, and Interpretation

The population-reweighting formulas follow from conditioning on \(R=1\), provided \(P(R=1)>0\). They do not require a causal model. For a causal interpretation of \(\tau_{\mathcal{S}}\), however, the propensity score must be based on relevant pretreatment covariates, and the causal assumptions must apply in the retained population. Conditional exchangeability and consistency are stated above; positivity requires a nonzero probability of each treatment level for covariate profiles in the target population. An interval strictly inside \((0,1)\) ensures this condition for the true propensity score among retained profiles.

Useful diagnostics should make the population change visible rather than report only how many units remain. At minimum, compare the retained fraction with the original population and compare important covariate summaries before and after restriction. For categorical covariates, report prevalences or group shares; for quantitative covariates, compare means and, where useful, distributions. Also describe which score-defined profiles were removed. A retained sample can be large while still differing substantially in composition from the original population.

If retention is determined using estimated rather than true propensity scores, the calculated retained fraction and covariate summaries describe the sample selected by that fitted score and rule. They may not exactly equal the population quantities in the derivation. A misspecified score model, limited data in some covariate profiles, or small changes in fitted scores near a region boundary can affect which units are retained. It is therefore important to document that selection is score-based and to check whether the resulting retained group is substantively interpretable.

Trimming is not the only way to address limited overlap. Analysts may instead use weighting or matching approaches that retain a broader population, but those approaches can rely more heavily on observations from poorly supported regions. The trade-off is not simply “biased” versus “unbiased”: the approaches can target different populations and have different sensitivity to sparse information. Restriction can make a target more focused, but it cannot justify describing a retained-population effect as an effect for everyone.

Common Mistakes and Limitations

• Using the wrong denominator. A retained mean is not the retained-group total divided by the original population size. Normalize by \(P(R=1)\), or by the retained count in a sample.
• Reporting only the percentage removed. A retained fraction does not reveal which profiles remain. Two retained populations of equal size can have different covariate distributions.
• Assuming the effect is unchanged. A different covariate mix can change the average treatment effect when effects vary across profiles. Equality with the full-population effect needs justification.
• Calling restriction a cure for confounding. Restriction changes the target and may improve support, but causal interpretation still depends on consistency, exchangeability, and positivity for the retained target.
• Overgeneralizing the conclusion. An effect for \(R=1\) describes units satisfying the score rule, not automatically excluded units or the original population as a whole.

A further limitation is substantive: the retained group may be a less relevant target for a policy or clinical decision if the excluded profiles matter to that decision. The restriction can also reduce the number of observations and therefore the precision of an estimate. Whether these costs are acceptable depends on the scientific question; a statistical convenience alone does not define a useful target population.