Overlap Weighting In A Population With Poor Positivity

Why target the overlap population?

When treatment is rare for some covariate profiles and nearly certain for others, the data provide little information about what would have happened under the less common treatment for people in those profiles. Inverse-probability weights for the average treatment effect can become very large in this setting. Trimming observations is one possible response, but it discards people according to a chosen cutoff. Overlap weighting instead gives greatest weight to people whose measured characteristics make either treatment plausible, and progressively less weight to people near the extremes.

This changes the question being answered. Rather than estimating an effect for the entire source population, overlap weighting targets an effect for a population concentrated where treated and untreated people overlap in their measured covariates. The result may be useful and stable, but it is not automatically an estimate of the average treatment effect (ATE) or the average treatment effect on the treated (ATT).

Definition: Let $A\in\{0,1\}$ indicate treatment, let $X$ denote measured pretreatment covariates, and let $e(X)=P(A=1\mid X)$ be the propensity score. The overlap weight is $1-e(X)$ for a treated individual and $e(X)$ for an untreated individual. The overlap population weights the source population in proportion to $e(X)\{1-e(X)\}$. Its average treatment effect is $$ \tau_{\mathrm{OW}}= \frac{E\left[e(X)\{1-e(X)\}\{Y(1)-Y(0)\}\right]} {E\left[e(X)\{1-e(X)\}\right]}. $$ Here $Y(1)$ and $Y(0)$ are the potential outcomes under treatment and no treatment.

The propensity score is estimated from the observed treatment and covariates; in practice, the weights use an estimated score, often written $\hat e(X)$. The overlap weights are bounded between zero and one, unlike the inverse-probability weights $1/\hat e(X)$ and $1/\{1-\hat e(X)\}$. Bounded weights limit the leverage of extreme estimated scores, but do not by themselves guarantee a good propensity model, adequate sample size, or causal identification.

Assumptions and what the estimate identifies

A causal interpretation requires more than calculating weights. Consistency means the observed outcome equals the potential outcome under the treatment actually received. No interference means one person's treatment does not change another person's potential outcome. Conditional exchangeability requires $\{Y(1),Y(0)\}$ to be independent of $A$ given $X$; in words, the measured covariates must suffice to control confounding. These assumptions cannot be established by examining weighted balance alone.

For the full source population, positivity requires $0<e(X)<1$ for covariate profiles in that population. With poor positivity, some profiles may have scores close to zero or one, so the data contain very few people in one treatment group. The overlap target places little weight on those profiles. For the overlap effect to be identified, there must still be information about both treatments in the covariate region receiving positive target weight, and the causal assumptions must hold there. If a profile has a true score exactly zero or one, its overlap weight is zero; no data-based procedure can identify a treatment contrast for that profile from this cohort.

The key distinction is that overlap weighting changes the target rather than manufacturing comparisons in unsupported regions. A well-behaved estimate can therefore describe a narrower population than clinicians or decision-makers initially had in mind. The target should be reported explicitly.

Weighted estimates, balance, and effective sample size

With estimated scores, calculate the weighted outcome mean separately in each treatment group. For an observed sample of $n$ individuals, the overlap-weighted estimate is the treated weighted mean minus the untreated weighted mean:

\hat\tau_{\mathrm{OW}}= \frac{\sum_{i=1}^{n} A_i\{1-\hat e(X_i)\}Y_i} {\sum_{i=1}^{n} A_i\{1-\hat e(X_i)\}} - \frac{\sum_{i=1}^{n} (1-A_i)\hat e(X_i)Y_i} {\sum_{i=1}^{n} (1-A_i)\hat e(X_i)}.

A weighted mean of a covariate is calculated using the same group-specific weights. Compare those weighted means, and also inspect distributions or other relevant summaries: equal means do not establish balance in all features of a distribution. The effective sample size (ESS) summarizes weight concentration within a group. For weights $w_i$, it is

\mathrm{ESS}=\frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^2}.

The ESS is at most the number of observations in that group; it is smaller when a few individuals carry a large share of total weight. It is a diagnostic, not a literal count of independent people and not a substitute for an uncertainty calculation.

Justification for the target: Given $X$, the expected treated overlap weight is $E[A\{1-e(X)\}\mid X]=e(X)\{1-e(X)\}$. The expected untreated overlap weight is $E[(1-A)e(X)\mid X]=\{1-e(X)\}e(X)$, the same quantity. Under consistency and conditional exchangeability, the weighted treated outcome mean therefore averages $Y(1)$ over the overlap population, and the weighted untreated mean averages $Y(0)$ over that same population. Their difference identifies $\tau_{\mathrm{OW}}$, provided the required treatment comparisons are supported in that target population.

Worked calculations

Worked Example: Estimating an Effect and Its ESS

Suppose a small hypothetical cohort has four treated and four untreated individuals. The listed propensity scores are supplied by a fitted model. Outcomes are days to recovery; lower values mean faster recovery.

Group	Propensity scores	Outcomes (days)
Treated	0.20, 0.40, 0.60, 0.80	10, 12, 14, 20
Untreated	0.20, 0.40, 0.60, 0.80	8, 9, 10, 11

The treated weights are $1-\hat e$, or $0.80, 0.60, 0.40, 0.20$. Their sum is $2.00$, and their weighted outcome sum is $0.80(10)+0.60(12)+0.40(14)+0.20(20)=24.8$. Thus the treated weighted mean is $24.8/2.00=12.4$ days.

The untreated weights are $\hat e$, or $0.20, 0.40, 0.60, 0.80$. Their sum is $2.00$, and their weighted outcome sum is $0.20(8)+0.40(9)+0.60(10)+0.80(11)=20.0$. The untreated weighted mean is $20.0/2.00=10.0$ days. Therefore, the estimated overlap-population treatment effect is $12.4-10.0=2.40$ days longer to recovery under treatment in this hypothetical cohort.

For the treated group, the sum of squared weights is $0.80^2+0.60^2+0.40^2+0.20^2=1.20$, so its ESS is $2.00^2/1.20=3.33$. The untreated group has the same squared-weight sum and ESS. Each group began with four individuals, but the weighting has an ESS of about 3.33. This calculation does not provide a confidence interval; uncertainty must be estimated with a method appropriate to the design and fitted propensity scores.

Worked Example: Checking Balance and ESS for a Binary Covariate

Consider a hypothetical sample with a binary covariate $X$, where $X=1$ denotes a particular risk factor. A logistic propensity model includes an intercept and $X$. There are 2 treated and 8 untreated people with $X=0$, and 8 treated and 2 untreated people with $X=1$. With this saturated model, the fitted scores within the two covariate groups are $2/10=0.20$ and $8/10=0.80$, respectively.

Before weighting, the treated proportion with $X=1$ is $8/10=0.80$; the untreated proportion is $2/10=0.20$. These groups differ substantially on this covariate. For $X=0$, the treated total overlap weight is $2(1-0.20)=1.60$, and the untreated total is $8(0.20)=1.60$. For $X=1$, the treated total is $8(1-0.80)=1.60$, and the untreated total is $2(0.80)=1.60$.

Each group's total weight is $3.20$, of which $1.60$ is attached to $X=1$. Thus both weighted proportions with $X=1$ are $1.60/3.20=0.50$: the covariate is balanced in weighted means. For treated people, the squared weights sum to $2(0.80^2)+8(0.20^2)=1.60$, giving $\mathrm{ESS}=3.20^2/1.60=6.40$. The untreated ESS is also $8(0.20^2)+2(0.80^2)=1.60$, hence 6.40. Here a fitted logistic model with an intercept and the covariate yields exact weighted mean balance for that covariate. It does not establish balance on variables or functional forms omitted from the model.

Worked Example: Describing the Overlap Target

Suppose a hypothetical population has three equally common covariate profiles, with propensity scores $0.05$, $0.50$, and $0.95$. Suppose the treatment-minus-control differences in mean recovery time within those profiles are 1, 4, and 10 days, respectively. For equally common profiles, their unnormalized overlap masses are proportional to $e(1-e)$: $0.05(0.95)=0.0475$, $0.50(0.50)=0.25$, and $0.95(0.05)=0.0475$. The total is $0.345$, so the normalized target shares are approximately $0.1377$, $0.7246$, and $0.1377$.

The overlap effect is therefore $0.1377(1)+0.7246(4)+0.1377(10)=4.413$ days. The middle profile contributes most because treatment is comparatively plausible both ways there. This is not the equally weighted average of the three profile effects, which would be $(1+4+10)/3=5.00$ days. The overlap-weighted result describes a population tilted toward profiles with treatment overlap, not the original equally distributed population.

Diagnostics, alternatives, and limitations

Before interpreting an estimate, examine the propensity-score distributions by treatment group and the resulting weights. Look for near-zero or near-one scores, regions represented in only one group, extreme imbalance in important covariates, very small group-specific ESS, and a weighted target that differs sharply from the population of scientific interest. Overlap weights cannot recover outcomes for an unsupported group, correct unmeasured confounding, or compensate for a badly specified propensity model.

A useful balance diagnostic compares treated and untreated covariate distributions after weighting. Standardized mean differences are common for continuous and binary covariates; inspect nonlinear terms and interactions that matter scientifically, not only the covariates' original means. If a logistic model is used, exact balance applies to the terms included in that model under the relevant fitting conditions; it is not a promise of balance for every covariate feature. Reconsider the model and target if important imbalance remains.

The main alternatives answer different questions or make different trade-offs. Inverse-probability weighting for the ATE targets the full source population but can be unstable when scores are extreme. ATT weighting emphasizes the treated population and requires sufficient untreated support for treated profiles. Matching and subclassification construct comparisons through pairs or score groups and may exclude or coarsen parts of the cohort. Trimming removes observations outside a chosen support region, making the retained target depend on the trimming rule. Overlap weighting avoids a hard cutoff and bounds weights, but deliberately changes the target to the overlap population.

An uncertainty interval should reflect the analysis design and the fact that propensity scores are estimated; treating fitted weights as fixed can give an inappropriate standard error. Report the score model, weighted balance, group-specific ESS, outcome scale, effect estimate and uncertainty, and a clear description of the overlap target. Do not describe the estimate simply as “the treatment effect” when the target population has changed.

Common mistake

Wrong reasoning: “The overlap weights are bounded, so positivity and confounding are no longer concerns, and the estimate is the ATE.”
Correction: Bounded weights reduce the influence of extreme scores, but the estimate targets people with treatment overlap, not necessarily the full population. Causal interpretation still depends on consistency, no interference, conditional exchangeability, and adequate treatment support in the target region.

Key takeaway: Overlap weighting can provide a stable comparison where treatment assignment is genuinely uncertain, while making the target population explicit: those with measured characteristics that give them a greater chance of receiving either treatment.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A treated person has estimated propensity score $0.90$. What overlap weight is used? What weight is used for an untreated person with the same score?
Why can overlap weighting have a different effect estimate from ATE weighting even when both use the same cohort?
A group has weights $0.5, 0.5, 1.0$. Calculate its ESS.
Name two diagnostics that should be reported alongside an overlap-weighted estimate.

Solutions. (1) The treated person's weight is $1-0.90=0.10$; the untreated person's weight is $0.90$. (2) The methods target different populations: overlap weighting emphasizes profiles with propensity scores nearer $0.50$, whereas the ATE targets the source population under its positivity assumptions. (3) The weight sum is $2.0$ and the squared-weight sum is $0.25+0.25+1.00=1.50$, so $\mathrm{ESS}=2.0^2/1.50=2.67$. (4) Appropriate answers include weighted covariate balance, propensity-score distributions by treatment, group-specific ESS, and the extent of weight concentration.