Why Two Inverse-Probability Estimators?
Inverse-probability weighting uses each unit’s treatment probability to reweight observed outcomes. For a target population of \(n\) units, the Horvitz–Thompson estimator scales each weighted outcome total by \(n\). The Hájek estimator instead divides the weighted outcome total by the realized total weight in that treatment arm. Both use the same inverse-probability weights, but their different denominators can produce different estimates in a finite sample.
The distinction matters when the realized sum of weights in an arm differs substantially from its target. Horvitz–Thompson estimation preserves the target population scale, which gives it a useful unbiasedness property when treatment probabilities are known. Hájek normalization adjusts the scale to the observed weight total; this often makes estimates less sensitive to unusually large or small realized totals, but its ratio form generally does not retain exact finite-sample unbiasedness.
These equations use \(e_i\) for the true propensity score. In practice it is often replaced by an estimate \(\hat e_i\), provided \(0<\hat e_i<1\). The formulas then have \(\hat e_i\) in place of \(e_i\). That substitution is important: the exact finite-sample unbiasedness result below assumes known assignment probabilities, not merely probabilities estimated from the same data.
Assumptions and the Unbiasedness Result
For a causal interpretation of an ATE, let \(Y_i(1)\) and \(Y_i(0)\) denote unit \(i\)’s potential outcomes under treatment and control. The observed outcome satisfies consistency, \(Y_i=A_iY_i(1)+(1-A_i)Y_i(0)\). Identification from inverse-probability weighting also requires conditional exchangeability, \((Y_i(1),Y_i(0))\perp A_i\mid X_i\), and positivity, \(0<e_i<1\) for covariate values in the target population. These conditions allow treatment assignment to be addressed using the measured covariates; they do not guarantee that an estimated propensity model is correctly specified.
A separate, more specific statement concerns finite-sample unbiasedness. Treat the covariates and potential outcomes of the \(n\) units as fixed, and suppose treatment is assigned with known marginal probabilities \(e_i=P(A_i=1\mid X_i)\). Then the Horvitz–Thompson mean for treatment is unbiased over the assignment process for the finite-population mean \(n^{-1}\sum_iY_i(1)\). The analogous claim holds for control and therefore for their difference. This result does not require independent treatment assignments across units; it follows from the correct marginal assignment probabilities.
Because consistency gives \(Y_i=Y_i(1)\) when \(A_i=1\), and \(Y_i(1)\) is fixed in this finite-population argument, \(E[A_iY_i/e_i]=Y_i(1)E[A_i]/e_i=Y_i(1)\).
Linearity of expectation gives \(E[\hat{\mu}_{1}^{HT}]=n^{-1}\sum_iY_i(1)\). The same reasoning, using \(P(A_i=0\mid X_i)=1-e_i\), gives \(E[\hat{\mu}_{0}^{HT}]=n^{-1}\sum_iY_i(0)\).
It follows that \(E[\hat{\tau}^{HT}]=n^{-1}\sum_i\{Y_i(1)-Y_i(0)\}\), the finite-population ATE.
The Hájek estimator is a ratio with a random denominator, so expectation does not generally pass through the ratio. Its denominator may also be zero if no units are observed in an arm. When the denominator is positive, the estimate is a weighted average of observed outcomes in that arm, if the weights are positive. Thus it lies between that arm’s smallest and largest observed outcomes. A Horvitz–Thompson mean need not: it divides the weighted total by \(n\), even when the realized arm-weight total is much larger or smaller than \(n\).
Worked Example: Different Estimates From the Same Weights
Consider five hypothetical units with the following observed treatment assignments, outcomes, and known propensity scores. The treatment-arm weight is \(1/e_i\); the control-arm weight is \(1/(1-e_i)\).
| Unit | \(e_i\) | \(A_i\) | Observed \(Y_i\) |
|---|---|---|---|
| 1 | 0.20 | 1 | 10 |
| 2 | 0.50 | 1 | 20 |
| 3 | 0.50 | 0 | 8 |
| 4 | 0.50 | 0 | 12 |
| 5 | 0.80 | 0 | 15 |
For treatment, the weighted outcome total is \(10/0.20+20/0.50=50+40=90\), and the sum of treatment weights is \(1/0.20+1/0.50=5+2=7\). Therefore,
For control, the weighted outcome total is \(8/0.50+12/0.50+15/0.20=16+24+75=115\). Its weight total is \(2+2+5=9\), so
The resulting ATE estimates are \(\hat{\tau}^{HT}=18-23=-5\) outcome units and \(\hat{\tau}^{H}\approx12.86-12.78=0.0794\) outcome units. The difference is driven by the realized weight totals: 7 for treatment and 9 for control, rather than the target total of 5 for each arm. In particular, the high control weight for unit 5 makes the Horvitz–Thompson control mean exceed every observed control outcome. This small hypothetical dataset illustrates the estimators’ arithmetic, not evidence that either estimate is a reliable causal conclusion.
Why the Hájek Ratio Can Be Biased
The ratio’s behavior can be seen in a small assignment example. Suppose two units have fixed treatment potential outcomes \(Y_1(1)=0\) and \(Y_2(1)=10\), with known treatment probabilities \(e_1=0.20\) and \(e_2=0.80\). The finite-population treated mean is \((0+10)/2=5\). The Horvitz–Thompson estimator is
Its expectation is \(6.25(0.80)=5\). The Hájek estimate is undefined if neither unit is treated; that event has probability \((0.80)(0.20)=0.16\). Conditional on observing at least one treated unit, there are three possible assignments. Only unit 1 is treated with probability \(0.04\), only unit 2 with probability \(0.64\), and both are treated with probability \(0.16\). The corresponding Hájek means are \(0\), \(10\), and \((0+10/0.80)/(1/0.20+1/0.80)=2\). Thus, conditional on a nonempty treatment arm, its expected value is
This conditional expectation differs from the target mean of 5. The example demonstrates two features of a ratio estimator: it can be undefined for some samples, and even when defined it need not be unbiased. In larger samples, with adequate positivity and regularity conditions, the Hájek estimator is generally consistent as the random weight total approaches its target. Consistency is not a claim of exact finite-sample unbiasedness.
Worked Example: When Normalization Changes Nothing—and When It Does
Suppose a hypothetical study has \(n=4\) units, and the two observed treated units have inverse-probability weights 2 and 1 and outcomes 4 and 10. The treatment-weighted outcome total is \(2(4)+1(10)=18\), while the treatment-weight total is \(2+1=3\). The two estimates are
To express the same Hájek calculation using weights that sum to \(n=4\), multiply both weights by \(4/3\). The rescaled weights are \(8/3\) and \(4/3\), and their weighted mean is
Rescaling every weight in one arm by the same positive constant leaves its Hájek mean unchanged, because the constant multiplies both numerator and denominator. If instead the original weights had summed to \(n=4\), the Horvitz–Thompson and Hájek means would coincide for that arm. They can differ precisely because the realized sum of weights is not generally equal to its target.
Diagnostics, Comparisons, and Limitations
Neither estimator repairs a lack of overlap or an inadequate propensity model. Very small estimated probabilities create very large inverse weights; a few units can then dominate the weighted total. Inspect propensity-score distributions by treatment group, the largest weights, and the arm-specific weight totals. Also assess covariate balance after weighting: reasonable-looking estimates do not establish that measured confounders have been adequately balanced. A large gap between the Horvitz–Thompson and Hájek results is a useful prompt to investigate realized weight totals and influential observations, not a formal test of model validity.
The weight totals have a direct diagnostic interpretation. With known true propensities, the expected treatment-weight total is \(n\), since \(E[A_i/e_i]=1\) for each unit; the expected control-weight total is also \(n\). The realized totals vary around these targets. Estimated propensities can further change their behavior, particularly when the fitted model produces extreme values. A total far from \(n\) can explain why the estimators differ, but it does not alone prove that one estimator is wrong.
In broad terms, Horvitz–Thompson estimation keeps the target-population scale explicit and has a design-unbiasedness result under known assignment probabilities. Its finite-sample estimates can be highly variable or outside the observed outcome range. Hájek estimation uses a data-dependent scale, is invariant to multiplying all weights in an arm by a constant, and produces a weighted mean within the arm’s observed outcome range. Its ratio form may have finite-sample bias and can fail when an arm has no observations. Neither estimator is always more efficient: their relative variance depends on the assignment process, outcomes, and weights.
Both are alternatives to approaches such as matching or outcome-model standardization, which use different calculations and can have different finite-sample behavior. Matching may change which units contribute to the comparison, while an outcome model relies on its own modeling assumptions. These alternatives do not make the weighting assumptions disappear. If propensity scores are estimated, both weighted estimators can also inherit error from that estimation, so the known-probability unbiasedness argument cannot simply be applied to them.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For a treatment arm with weighted outcome total 84, weight total 12, and target size \(n=10\), calculate the Horvitz–Thompson and Hájek means. Explain the difference.
- Under what conditions does the finite-sample unbiasedness result for the Horvitz–Thompson treated mean apply? Does using an estimated propensity score automatically meet those conditions?
- Why might a Hájek mean be undefined, and why is it bounded by observed outcomes when it is defined and uses positive weights?
- A weighted analysis has a treatment-weight total much larger than \(n\), and its Horvitz–Thompson estimate is larger than every observed treated outcome. Is this necessarily an arithmetic error? Explain.
Solutions. (1) The Horvitz–Thompson mean is \(84/10=8.4\); the Hájek mean is \(84/12=7\). The numerators agree, but Hájek divides by the realized weight total, which here exceeds \(n\). (2) The argument treats the covariates and potential outcomes as fixed and uses the true known marginal treatment probabilities \(e_i\). Then \(E[A_iY_i(1)/e_i]=Y_i(1)\) for each unit. An estimated propensity score does not automatically satisfy the known-probability condition, so exact finite-sample unbiasedness does not follow. (3) If no treated units are observed, the treatment weight total is zero and the ratio is undefined. Otherwise, with positive weights, the Hájek mean is a weighted average: its weights after division by their sum are nonnegative and sum to one, so the result lies between the smallest and largest observed treated outcomes. (4) No. The Horvitz–Thompson estimate divides by \(n\), not by the realized treatment-weight total. A total larger than \(n\) can therefore produce a mean outside the observed outcome range. Check the weights and calculations, and investigate extreme weights and overlap before interpreting the result.