From Propensity Scores to an Overlap-Weighted Effect
A propensity score is the probability of receiving treatment given measured covariates. Overlap weighting uses that probability to give relatively more weight to people whose covariate profiles make either treatment plausible. A compact dataset makes each part of the calculation visible: the scores, the individual weights, the weighted outcome means, and the population those means describe.
Let \(A_i\) equal 1 for a treated unit and 0 for a control unit; let \(X_i\) denote measured pre-treatment covariates; and let \(Y_i\) be the observed outcome. Write \(Y_i(1)\) and \(Y_i(0)\) for the potential outcomes under treatment and control. The propensity score is \(e(X_i)=P(A_i=1\mid X_i)\). For this tutorial, outcomes are numerical, and the target effect is a difference in mean outcomes.
The factor \(e(X)\{1-e(X)\}\) is largest at \(e(X)=0.5\) and approaches zero as the score approaches 0 or 1. Thus, the target population emphasizes covariate profiles with treatment overlap; it is not automatically the full study population, the treated population, or the control population.
Assumptions and the Hand-Calculation Setup
A causal interpretation requires the usual conditions for identifying effects from measured covariates. Consistency means the observed outcome equals the potential outcome under the treatment actually received. No interference means one unit’s treatment does not change another unit’s outcome. <strongConditional exchangeability requires \(\{Y(1),Y(0)\}\) to be independent of \(A\) conditional on \(X\). Finally, positivity requires \(0<e(X)<1\) for covariate profiles represented in the target population. These conditions cannot be verified from the arithmetic alone.
The dataset below is hypothetical. It has one categorical covariate \(X\) with three levels, 0, 1, and 2. The outcome totals are included so weighted means can be calculated without listing every individual outcome.
| Covariate level \(X\) | Treated count \(n_1\) | Control count \(n_0\) | Treated outcome sum | Control outcome sum |
|---|---|---|---|---|
| 0 | 2 | 8 | 22 | 84 |
| 1 | 5 | 5 | 70 | 60 |
| 2 | 8 | 2 | 148 | 30 |
Worked Example: Calculate the Propensity Scores
Within each covariate level, estimate the treatment probability as the treated fraction. Equivalently, a saturated logistic model for the three-level categorical covariate estimates a separate probability at each level. Here, the observed treatment proportions are the fitted scores.
At \(X=0\), there are 2 treated and 8 controls, so \(e(0)=2/(2+8)=0.2\). At \(X=1\), \(e(1)=5/(5+5)=0.5\). At \(X=2\), \(e(2)=8/(8+2)=0.8\).
These probabilities can also be represented by the logistic model \(\operatorname{logit}\{e(x)\}=-\log(4)+x\log(4)\), for these three levels. For example, at \(x=2\), the log odds are \(-\log(4)+2\log(4)=\log(4)\), so the odds are 4 and the probability is \(4/(1+4)=0.8\). The coefficient is exactly \(\log(4)\), not a rounded approximation. The fitted probabilities therefore agree with the treatment fractions shown in the table.
All three estimated scores lie strictly between 0 and 1, so there is empirical treatment and control representation at each level. This is a useful finite-sample check, but it does not prove that exchangeability holds or that the model captures every relevant covariate.
Apply the Weights and Calculate the Effect
For each treated unit, use \(1-e_i\); for each control unit, use \(e_i\). The weighted mean outcome in each treatment arm is the sum of the unit weights times observed outcomes in that arm, divided by the sum of the weights in that arm. With grouped data, multiply each group’s weight by its count for the denominator, and by its outcome sum for the numerator.
Worked Example: Calculate the Group Weights and Weighted Means
At \(X=0\), treated units receive weight \(1-0.2=0.8\), and controls receive weight \(0.2\). The total treated weight is \(2(0.8)=1.6\), and the total control weight is \(8(0.2)=1.6\). At \(X=1\), each unit receives weight \(0.5\), giving total weight \(5(0.5)=2.5\) in each arm. At \(X=2\), treated units receive weight \(0.2\), while controls receive weight \(0.8\); each arm’s total weight is \(1.6\).
Thus, the total weight is \(1.6+2.5+1.6=5.7\) in each arm. The treated weighted outcome total is
The treated weighted mean is \(82.2/5.7=14.4211\) outcome units. The control weighted outcome total and mean are
Subtract the control weighted mean from the treated weighted mean: \(14.4211-12.4211=2.000\) outcome units. Under the identification assumptions, this estimates the average treatment effect in the overlap population. It is not, merely by virtue of being a weighted contrast, an estimate of the ATE for the full original population.
Why the Weights Balance Covariates
The hand calculation illustrates the balance property. In a covariate group with treatment probability \(e(x)\), the expected treated covariate density after weighting is proportional to \(e(x)\{1-e(x)\}f(x)\): the treated density contributes a factor \(e(x)\), and the treated weight contributes \(1-e(x)\). For controls, the control density contributes \(1-e(x)\), and the control weight contributes \(e(x)\). The product is the same.
In the constructed table, the equality is exact at each covariate level because the fitted scores equal the observed treatment fractions there. For \(X=0\), for instance, the treated total weight is \(2(0.8)=1.6\), and the control total weight is \(8(0.2)=1.6\). The same equality holds at levels 1 and 2. This finite-sample balance in the included covariate is a consequence of the setup and score fit; it does not establish balance on omitted covariates.
For the example, the normalized covariate-group weights are \(1.6/5.7\), \(2.5/5.7\), and \(1.6/5.7\), or approximately 0.2807, 0.4386, and 0.2807. These proportions are the same in both treatment arms. They also show that the target puts the greatest relative emphasis on \(X=1\), where treatment was equally common.
Check the Calculation and Recognize Failure Modes
Worked Example: Diagnose a Score Near the Edge
Suppose a different hypothetical dataset has fitted scores near 0.02 for one covariate profile and 0.98 for another. At the first profile, a treated unit’s overlap weight is \(1-0.02=0.98\), while a control unit’s weight is \(0.02\). At the second profile, a treated unit’s weight is \(1-0.98=0.02\), and a control unit’s weight is \(0.98\).
The weights are bounded between 0 and 1, so they do not become enormous in the way some inverse-probability weights can. However, the data may contain very few units from the less common treatment arm in either profile. A small number of observations can then carry much of an arm’s total overlap weight, making estimates sensitive to individual outcomes. Inspect score distributions by treatment, group counts, weighted covariate balance, and the contribution of individual observations. A finite score alone does not guarantee a precise or credible estimate.
A practical diagnostic is to compare covariate distributions between weighted treatment arms. For categorical covariates, compare the weighted proportions in each level; for quantitative covariates, compare weighted means and distributions. A large remaining difference can indicate score-model misspecification, coding errors, or a mismatch between the weighting calculation and the intended method. Also verify that the treatment indicator is coded consistently and that scores are probabilities between 0 and 1.
The overlap estimate depends on the estimated propensity scores and on which covariates were included. If an important confounder is omitted, excellent balance on included variables cannot remove that unmeasured confounding. If treatment is deterministic for a covariate profile, the positivity condition fails there; overlap weighting places essentially no target weight on that extreme profile, but it cannot recover an effect for a population requiring unsupported treatment comparisons.
How This Estimand Differs From Alternatives
Inverse-probability weights for the ATE use \(1/e_i\) for treated units and \(1/(1-e_i)\) for controls, targeting the original population under the relevant assumptions. ATT and ATU weighting target the treated and untreated populations, respectively. Overlap weights instead target a covariate distribution concentrated where either treatment is plausible. Matching also aims to construct comparable groups, but its estimand and retained sample depend on the matching design and which units are matched. These methods answer different population questions; one should not select between them solely by which estimate looks more favorable.
A common mistake is to subtract weighted means and call the result “the ATE” without specifying the target. In this example the arithmetic contrast is 2.000 outcome units, but its target is the overlap population. Another error is to claim that balance on the modeled covariates proves a causal effect. Balance is a diagnostic for measured covariates; causal interpretation still requires the identification assumptions, including no unmeasured confounding conditional on \(X\).
Check Your Understanding
Answer each question, then compare with the solutions below.
- In a covariate group with 3 treated and 7 controls, what is the empirical propensity score, and what overlap weight does each arm receive?
- For that group, verify that the total treated and control weights are equal.
- Suppose a treated weighted mean is 9.5 points and a control weighted mean is 7.0 points. What is the estimated contrast, and what population does it target?
- Why does exact weighted balance on a modeled covariate not establish that the causal assumptions hold?
- What diagnostic concern arises when a profile has propensity score 0.99, even though overlap weights cannot exceed 1?
Solutions. (1) The score is \(3/(3+7)=0.3\). Treated units receive \(1-0.3=0.7\), and controls receive 0.3. (2) Treated total weight is \(3(0.7)=2.1\); control total weight is \(7(0.3)=2.1\). (3) The contrast is \(9.5-7.0=2.5\) outcome points, targeting the overlap population. (4) It checks balance only for the modeled measured covariate; it does not establish consistency, exchangeability, or absence of unmeasured confounding. (5) Few controls may occur at that profile, so the effective information can be limited and estimates may be sensitive to a small number of observations; inspect group counts, score distributions, and influence.