Common Errors in Inverse Probability Weight Construction

Why Weight Construction Deserves a Separate Check

Inverse probability weighting can look mechanical: estimate a propensity score, calculate weights, then compare weighted outcomes. But a single reversed indicator or mismatched denominator can change the result substantially. Even perfectly calculated arithmetic cannot rescue weights that target a different population from the one named in the analysis.

This tutorial focuses on checking the construction and use of weights. Let \(A\) be a binary treatment indicator, coded \(1\) for treated and \(0\) for control; let \(X\) denote measured pretreatment covariates; let \(Y\) be the observed outcome; and let \(e(X)=P(A=1\mid X)\) be the propensity score. The potential outcomes are \(Y(1)\) and \(Y(0)\), and consistency means \(Y=AY(1)+(1-A)Y(0)\).

Assumptions for the causal interpretations below: Consistency; conditional exchangeability, \((Y(1),Y(0))\perp A\mid X\); and positivity, \(0<e(X)<1\) for covariate values in the target population. In practice, the propensity score is estimated, so the fitted scores must also adequately represent treatment assignment given the measured covariates. These assumptions support interpretation of weighted contrasts as causal effects; they do not guarantee that a particular finite-sample estimate is unbiased or precise.

Start With the Treatment Indicator and the Target

Under the stated coding, a treated person has \(A=1\) and propensity score \(e(X)\); a control person has \(A=0\) and probability \(1-e(X)\) of receiving their observed treatment. For an average treatment effect (ATE), the unstabilized inverse probability weight is the reciprocal of the probability of the treatment actually received.

w_i^{ATE}=\frac{A_i}{e(X_i)}+\frac{1-A_i}{1-e(X_i)} = \begin{cases} 1/e(X_i), & A_i=1,\\ 1/\{1-e(X_i)\}, & A_i=0. \end{cases}

The ATE is \(E[Y(1)-Y(0)]\). In a normalized, or Hájek, estimator, calculate a weighted outcome mean separately in each observed treatment arm and subtract the control mean from the treated mean. Each arm's denominator is the sum of that arm's weights; it is not a shared denominator.

\widehat{\mu}_1^{ATE} = \frac{\sum_i A_iY_i/e(X_i)}{\sum_i A_i/e(X_i)}, \qquad \widehat{\mu}_0^{ATE} = \frac{\sum_i (1-A_i)Y_i/\{1-e(X_i)\}}{\sum_i (1-A_i)/\{1-e(X_i)\}}, \qquad \widehat{\tau}_{ATE}=\widehat{\mu}_1^{ATE}-\widehat{\mu}_0^{ATE}.

The formula must change when the target changes. For the average treatment effect among the treated (ATT), treated people represent themselves and controls are weighted to represent the treated group. For overlap weighting, each arm is weighted toward a population concentrated among people with intermediate propensity scores.

Target	Treated-person weight	Control-person weight	Target population
ATE	\(1/e(X)\)	\(1/\{1-e(X)\}\)	The population represented by the analysis sample
ATT	1	\(e(X)/\{1-e(X)\}\)	People who received treatment
Overlap effect	\(1-e(X)\)	\(e(X)\)	An overlap population weighted in proportion to \(e(X)\{1-e(X)\}\)

The target column is part of the specification, not a label added after the calculation. A formula that is correct for one target can be wrong for another. The same is true of normalization: report which estimator is being used, then keep its formula and denominator consistent.

Worked Example: Catching a Reversed Treatment Indicator

Consider four hypothetical people with the following observed treatment, fitted propensity score, and outcome. The outcome is measured in points.

Person	\(A\)	\(e(X)\)	\(Y\) (points)
1	1	0.80	4
2	1	0.40	9
3	0	0.20	4
4	0	0.60	8

For an ATE analysis, people 1 and 2 are treated, so their weights are \(1/0.80=1.25\) and \(1/0.40=2.50\). People 3 and 4 are controls, so their weights are \(1/(1-0.20)=1.25\) and \(1/(1-0.60)=2.50\). The normalized weighted outcome means are

\widehat{\mu}_1^{ATE} =\frac{4(1.25)+9(2.50)}{1.25+2.50} =\frac{27.5}{3.75} =7.333\text{ points}, \qquad \widehat{\mu}_0^{ATE} =\frac{4(1.25)+8(2.50)}{1.25+2.50} =\frac{25}{3.75} =6.667\text{ points}.

The ATE estimate is about \(0.667\) points, calculated from the unrounded means as \((27.5/3.75)-(25/3.75)=0.6667\) points. If the rounded means are subtracted, \(7.333-6.667\approx0.666\) points; that rounded subtraction is not an exact equality yielding \(0.667\).

Now suppose the analyst mistakenly assigns \(1/\{1-e(X)\}\) to treated people and \(1/e(X)\) to controls. The treated weights become \(5.00\) and \(1.667\), giving a mean of \(\{4(5.00)+9(1.667)\}/(5.00+1.667)=5.250\) points. The control weights become \(5.00\) and \(1.667\), giving \(\{4(5.00)+8(1.667)\}/(5.00+1.667)=5.000\) points. The reversed construction produces a contrast of \(0.250\) points, not the correct ATE estimate above. The quick check is to ask whether the denominator of each individual's weight is the probability of the treatment that person actually received.

Denominators and Weights Must Belong to the Same Estimator

An arm-specific normalized weighted mean divides by that arm's sum of weights. Dividing by the number of people in that arm instead does not calculate the same estimator. There is also a distinct estimator, the Horvitz–Thompson estimator, which uses the total sample size \(n\) as the denominator for each estimated potential-outcome mean. These conventions should not be mixed or described interchangeably.

\widehat{\mu}_{1,HT} =\frac{1}{n}\sum_i\frac{A_iY_i}{e(X_i)}, \qquad \widehat{\mu}_{0,HT} =\frac{1}{n}\sum_i\frac{(1-A_i)Y_i}{1-e(X_i)}.

The Horvitz–Thompson form uses \(n\), the number of all sampled people, in both denominators. The Hájek form uses the corresponding arm's sum of weights. They are different finite-sample estimators; with valid identification conditions, both are used to estimate the same population means. A denominator error occurs when a calculation follows one form but uses the denominator from another—for example, dividing a Hájek numerator by the observed number in its treatment arm.

Worked Example: Finding a Denominator and an Omitted-Arm Error

Use the four hypothetical records in the previous example. For the normalized ATE estimate, the treated weighted outcome sum is \(27.5\) points and the treated weight total is \(3.75\); the control weighted outcome sum is \(25\) points and the control weight total is also \(3.75\). Thus the arm-specific normalized means are \(27.5/3.75=7.333\) and \(25/3.75=6.667\) points.

There are two treated and two control people. Dividing the same weighted sums by the corresponding arm counts gives \(27.5/2=13.75\) and \(25/2=12.50\) points. Those are not normalized weighted means. The subtraction \(13.75-12.50=1.25\) points therefore does not reproduce the Hájek contrast. If the analyst instead intends the Horvitz–Thompson form, the denominator for each arm-specific potential-outcome mean is the full sample size \(n=4\): \(27.5/4=6.875\) points and \(25/4=6.250\) points, for a contrast of \(0.625\) points. That is a different, explicitly defined estimator—not a reason to call \(n_t=2\) the correct denominator for the normalized means.

A separate mistake is to apply weights only to controls. The unweighted treated mean here is \((4+9)/2=6.50\) points, while the correctly normalized weighted control mean is \(6.667\) points. Subtracting gives \(-0.167\) points. This calculation mixes an unweighted mean for one arm with a weighted mean for the other; it is not the stated ATE weighting estimator. A useful audit is to inspect the treated and control contributions separately and verify that both arms receive the prescribed weights and normalization.

Weights Must Match the Estimand

The ATE, ATT, and overlap effect do not generally describe the same population average. Their weight formulas encode different target populations, so different estimates from the same data are not automatically conflicting results. The analyst should specify the target first and then apply the matching weights to both arms as required.

Worked Example: Comparing Target-Specific Weight Formulas

Return to the four hypothetical records. For the ATE, the calculated normalized means are \(7.333\) points for treated people and \(6.667\) points for controls, giving an estimate of \(0.6667\) points from the unrounded values.

For the ATT, treated weights are 1, so the treated mean is \((4+9)/2=6.50\) points. The two controls have \(e/(1-e)\) weights \(0.20/0.80=0.25\) and \(0.60/0.40=1.50\). Their weighted mean is

\widehat{\mu}_{0}^{ATT} =\frac{4(0.25)+8(1.50)}{0.25+1.50} =\frac{13}{1.75} =7.429\text{ points}.

The ATT contrast is \(6.50-7.429=-0.929\) points. Its negative value does not contradict the positive ATE estimate: the two estimates target different populations, and this very small hypothetical data set also has substantial sampling variability.

For overlap weighting, treated weights are \(1-e(X)\), while control weights are \(e(X)\). The treated weighted mean is \(\{4(0.20)+9(0.60)\}/(0.20+0.60)=6.20/0.80=7.75\) points. The control weighted mean is \(\{4(0.20)+8(0.60)\}/(0.20+0.60)=5.60/0.80=7.00\) points. The overlap-weighted contrast is \(0.75\) points. It describes the overlap target rather than the full analysis population or treated population.

Why the ATE Formula Works—and What Can Still Go Wrong

The weighting identity explains why the ATE weights are paired as shown. For a treated person, consistency gives \(Y=Y(1)\). Conditional exchangeability allows the treated outcome conditional on \(X\) to represent the potential outcome conditional on \(X\). By iterated expectation and the definition of \(e(X)\),

E\left[\frac{AY}{e(X)}\middle|X\right] = \frac{P(A=1\mid X)E[Y\mid A=1,X]}{e(X)} = E[Y(1)\mid X].

Averaging over \(X\) gives \(E[AY/e(X)]=E[Y(1)]\). The analogous control identity is \(E[(1-A)Y/\{1-e(X)\}]=E[Y(0)]\). The normalized estimator additionally divides by the observed arm-specific weight total, stabilizing the sample mean's scale. This argument depends on the assumptions stated earlier and on using the correct treatment probability for each observed arm; it does not establish that an estimated propensity model is adequate.

Common errors often leave clues. Check that all treated weights use \(1/e(X)\) for ATE and all control weights use \(1/\{1-e(X)\}\); verify each arm's weight total and weighted outcome numerator; and confirm that the calculation applies the intended normalization. Extremely large weights can indicate estimated probabilities near zero or one, making estimates unstable. This is a warning to investigate support and weight behavior, not proof by itself that a particular assumption has failed.

After construction, examine whether weighted covariate distributions are more comparable across treatment arms, since residual imbalance can reveal a poor propensity model or implementation error. Such balance checks do not prove exchangeability, which concerns measured and unmeasured causes of treatment and outcome. Subclassification and matching are alternative ways to adjust for measured covariates; unlike inverse probability weighting, they do not implement this same reciprocal-probability reweighting. Direct standardization also targets a specified covariate distribution, but calculates standardized outcomes rather than relying on these individual inverse weights. None of these alternatives removes the need to state the target population and assumptions.

Common Mistakes to Avoid

Reversing the indicator. Using \(1/\{1-e(X)\}\) for treated people assumes their observed-treatment probability is the control probability. Recheck the coding of \(A\) and the probability in every denominator.
Using an unexplained denominator. For normalized arm means, divide by that arm's sum of weights. For Horvitz–Thompson means, divide by the full sample size. Do not use an arm count while describing a normalized estimator.
Weighting only one arm. Applying the ATE weights to controls but leaving treated people unweighted creates a mixed contrast, not the stated two-arm ATE estimator.
Calling every weighting formula an ATE weight. ATT and overlap weights target different populations. Name the estimand before selecting the formula.
Treating balance as proof of causality. Balance diagnostics address observed covariates; they cannot verify the absence of unmeasured confounding.

Key takeaway: Audit a weighting analysis in this order: confirm treatment coding, write the target estimand, assign each arm its target-specific weight, use the matching denominator, and then check the weighted data and assumptions. A correct calculation for the wrong target is still the wrong analysis.

Check Your Understanding

Answer each question, then compare with the solutions below.

For an ATE analysis, what is the weight of a treated person whose propensity score is \(0.25\)?
For an ATE analysis, what is the weight of a control person whose propensity score is \(0.70\)?
Under ATT weighting, what are the treated and control weights when \(e(X)=0.60\)?
In a normalized weighted mean for treated people, should the denominator be the full sample size, the number treated, or the sum of treated weights?

Solutions. (1) A treated person receives \(1/e(X)=1/0.25=4\). (2) A control person receives \(1/\{1-e(X)\}=1/(1-0.70)=3.333\). (3) The treated weight is 1; the control weight is \(e(X)/\{1-e(X)\}=0.60/0.40=1.50\). (4) Use the sum of treated weights for a normalized weighted mean. The full sample size is used in the Horvitz–Thompson form, while the number treated is not the denominator for either of these stated estimators.