Why Use Overlap Weights?
Inverse probability weights can make a weighted comparison represent a chosen target population, but ATE weights can become very large when estimated propensity scores are close to zero or one. Such observations have little representation in the opposite treatment group, yet can have a large influence under inverse probability weighting. Overlap weights take a different approach: they give greater weight to units whose estimated propensity scores suggest that either treatment could plausibly have been received.
This tutorial focuses on the weights themselves and on calculating a weighted outcome contrast. The contrast is not automatically a causal effect: its causal interpretation still depends on the design and assumptions. The earlier material on the pseudo-population interpretation of weighting provides context for why weighted comparisons are used.
The weight is the complement of the estimated propensity score for treated units and the score itself for untreated units. Since \(0\leq\hat e_i\leq1\), each overlap weight lies between zero and one. A treated unit with \(\hat e_i=0.50\), for example, receives weight \(0.50\); a treated unit with \(\hat e_i=0.95\) receives weight \(0.05\). The latter has little weight because its estimated covariate profile is one for which treatment was highly likely.
These weights are generally not normalized to sum to one within each treatment group. To calculate a weighted mean, divide the weighted outcome total by that group’s total weight. For observed outcome \(Y_i\), the normalized overlap-weighted contrast is:
The estimator is a difference between two weighted means, with the treated mean first. Dividing by the total weight in each group is essential: the raw weighted outcome totals are not means. This normalization also makes the calculation invariant to multiplying all weights in one group by the same positive constant.
Assumptions Before Interpretation
The arithmetic of overlap weighting can be done whenever propensity scores and outcomes are available. Interpreting the resulting contrast as causal requires additional conditions. Let \(Y_i(1)\) and \(Y_i(0)\) denote unit \(i\)’s potential outcomes under treatment and no treatment, respectively. The observed outcome must satisfy consistency, \(Y_i=A_iY_i(1)+(1-A_i)Y_i(0)\), and treatment assignment must be conditionally exchangeable given measured covariates:
In words, after conditioning on the measured pretreatment covariates, treatment assignment must not carry additional information about the potential outcomes. This is an unconfoundedness assumption, not something established by the weighting calculation. A positivity condition requires both treatment options to have positive probability in covariate profiles relevant to the target comparison: \(0<e(X_i)<1\). Overlap weights reduce the influence of profiles with scores near the extremes, but they do not create observations from a missing treatment group or repair unmeasured confounding.
The estimated propensity scores must also be adequate for the data and the intended comparison. A misspecified score model can leave important covariate imbalance after weighting. The estimator uses \(\hat e_i\), so its behavior depends on how those scores were estimated; balance and score diagnostics are needed rather than assuming that a fitted model has succeeded.
Worked Example: Assigning Individual Overlap Weights
Suppose four hypothetical units have the following treatment assignments and estimated propensity scores. Compute each overlap weight and the total weight in each treatment group.
| Unit | \(A_i\) | \(\hat e_i\) | Overlap weight |
|---|---|---|---|
| A | 1 | 0.90 | \(1-0.90=0.10\) |
| B | 1 | 0.65 | \(1-0.65=0.35\) |
| C | 0 | 0.40 | \(0.40\) |
| D | 0 | 0.10 | \(0.10\) |
Units A and B are treated, so their weights are \(1-\hat e_i\): \(0.10\) and \(0.35\). Units C and D are untreated, so their weights are \(\hat e_i\): \(0.40\) and \(0.10\). Thus the total treated weight is \(0.10+0.35=0.45\), and the total untreated weight is \(0.40+0.10=0.50\). The totals need not be equal. Each group’s weighted mean uses its own total weight as the denominator.
The small weight for treated unit A reflects its high estimated probability of treatment. The small weight for untreated unit D reflects its low estimated probability of treatment. These calculations describe how the weighting scheme operates; they do not establish that the propensity model or causal assumptions are adequate.
Calculating a Weighted Outcome Contrast
For a transparent calculation, suppose a separate hypothetical dataset contains two treated and two untreated units. Their outcomes are measured in points. Assume the estimated propensity scores are as shown; for now, the task is to calculate the estimator, not to claim the assumptions have been verified.
Worked Example: A Treated-Minus-Untreated Contrast
| Unit | Treatment \(A_i\) | \(\hat e_i\) | Outcome \(Y_i\) (points) | Weight |
|---|---|---|---|---|
| T1 | 1 | 0.80 | 12 | 0.20 |
| T2 | 1 | 0.60 | 8 | 0.40 |
| C1 | 0 | 0.40 | 7 | 0.40 |
| C2 | 0 | 0.20 | 5 | 0.20 |
For the treated group, the weighted outcome total is \(0.20(12)+0.40(8)=2.40+3.20=5.60\), and its total weight is \(0.20+0.40=0.60\). Therefore:
For the untreated group, the weighted outcome total is \(0.40(7)+0.20(5)=2.80+1.00=3.80\), and its total weight is \(0.40+0.20=0.60\). Therefore:
The overlap-weighted treated-minus-untreated outcome contrast is 3.00 points. Here the two total weights happen to be equal, so the contrast can also be written as the difference between the weighted totals divided by \(0.60\). In general, calculate each group’s weighted mean with its own denominator before subtracting. This numerical contrast is descriptive of the weighting calculation; a causal interpretation requires the assumptions and diagnostics discussed above.
Why the Weights Can Help—and a Balance Check
The weight rule is designed to reduce the influence of observations whose scores indicate little treatment overlap. In particular, overlap weights remain bounded even when an estimated score is close to zero or one, unlike inverse probability weights that involve division by the score or its complement. Bounded weights can reduce sensitivity to a few very large weights. They do not guarantee good covariate balance, a well-estimated effect, or adequate information in both groups.
A basic diagnostic is to compare covariate distributions after weighting. For a binary covariate \(X_i\), calculate its weighted prevalence separately in each treatment group by replacing \(Y_i\) with \(X_i\) in the weighted-mean formula. The following hypothetical example shows how to do this and how to interpret the result.
Worked Example: Checking Weighted Covariate Balance
Suppose a binary covariate \(X\) is coded 1 when present and 0 otherwise. Four hypothetical units have these values:
| Unit | \(A_i\) | \(\hat e_i\) | \(X_i\) | Overlap weight |
|---|---|---|---|---|
| T1 | 1 | 0.75 | 1 | 0.25 |
| T2 | 1 | 0.25 | 0 | 0.75 |
| C1 | 0 | 0.25 | 1 | 0.25 |
| C2 | 0 | 0.75 | 0 | 0.75 |
Among treated units, the weighted prevalence of \(X=1\) is \(0.25(1)+0.75(0)=0.25\), divided by total treated weight \(0.25+0.75=1.00\). It is therefore \(0.25/1.00=0.250\). Among untreated units it is also \(0.25(1)+0.75(0)=0.25\), divided by total untreated weight \(1.00\), giving \(0.250\). The weighted prevalence difference is \(0.250-0.250=0\) percentage points on the proportion scale.
This is balance on this one covariate in this small example; it does not prove balance on other covariates or eliminate unmeasured confounding. In applied work, inspect all important pretreatment covariates and relevant features of their distributions, not only their means or one binary indicator.
Another useful diagnostic is the effective sample size (ESS), which summarizes how concentrated a set of weights is. For weights \(w_i\) in one treatment group, define:
For the treated weights \(0.25\) and \(0.75\) in the balance example, \(ESS=1.00^2/(0.25^2+0.75^2)=1/0.625=1.60\), below the actual group size of two. Weight concentration reduces the information represented by the weighted sample. ESS is a diagnostic, not a replacement for the sample size or a complete measure of estimator precision.
Comparison, Limitations, and Common Mistakes
| Weighting target | Treated-unit weight | Untreated-unit weight |
|---|---|---|
| ATE | \(1/\hat e_i\) | \(1/(1-\hat e_i)\) |
| ATT | 1 | \(\hat e_i/(1-\hat e_i)\) |
| ATU | \((1-\hat e_i)/\hat e_i\) | 1 |
| Overlap weights | \(1-\hat e_i\) | \(\hat e_i\) |
The methods differ in their weights and target comparisons. ATE, ATT, and ATU weighting, covered in the earlier material on those estimators, can assign large weights to observations with scores near zero or one. Overlap weights are bounded and emphasize units with more treatment overlap. This contrast is not a universal ranking: the appropriate choice depends on the scientific target and available data. The later derivation of the overlap population estimand gives the formal target interpretation of overlap weighting.
Common errors include treating the overlap weights as normalized when they are not, subtracting weighted totals instead of normalized means, or describing a weighted contrast as causal merely because the propensity score was estimated. Also avoid interpreting a weight near zero as evidence that the unit is irrelevant to every scientific question. It contributes little to this particular weighted comparison; another target population may call for a different weighting scheme.
- Check both denominators. The treated mean divides by the sum of treated weights; the untreated mean divides by the sum of untreated weights.
- Inspect weighted balance. A bounded weight does not guarantee that covariates are balanced after weighting.
- Check score distributions and ESS. Very low group-specific ESS or poor score overlap can signal limited information.
- Keep the causal assumptions explicit. Weighting measured covariates cannot correct confounding by variables that were not adequately measured and modeled.
A practical justification for normalization is immediate from a weighted mean: multiplying every treated weight by a common positive constant multiplies both its weighted outcome total and its total weight by that constant, leaving their ratio unchanged. The same holds for the untreated mean. Thus the contrast depends on relative weights within each group, not on whether the weights happen to sum to the same number across groups.
Check Your Understanding
Try each question, then compare your reasoning with the solutions below.
- A treated unit has estimated propensity score \(0.72\). What is its overlap weight? What would the weight be for an untreated unit with the same score?
- Two untreated units have scores \(0.25\) and \(0.75\), outcomes 4 and 10 points, respectively. One treated unit has score \(0.25\) and outcome 9 points. Calculate the overlap-weighted treated-minus-untreated contrast.
- Why does the fact that overlap weights are bounded not establish that a weighted contrast is causal?
- For treated weights \(0.25\) and \(0.75\), calculate the ESS. What does it indicate relative to the group size?
Solutions. 1. The treated weight is \(1-0.72=0.28\); the untreated weight is \(0.72\). 2. The treated mean is \(0.75(9)/0.75=9\) points. The untreated weights equal the scores, so the mean is \([0.25(4)+0.75(10)]/(0.25+0.75)=(1+7.5)/1=8.5\) points. The contrast is \(9-8.5=0.5\) points. 3. Bounded weights limit weight magnitude but do not verify conditional exchangeability, positivity in relevant profiles, or adequate propensity-score estimation and covariate balance. 4. \(ESS=(0.25+0.75)^2/(0.25^2+0.75^2)=1/0.625=1.60\). The two observations represent an effective sample size of 1.60 under these weights, reflecting some concentration rather than two equally weighted units.