Variance of the Inverse Probability Weighted Mean

Why a Weighted Mean Needs Its Own Variance Estimate

Inverse probability weighting changes both the contribution each observation makes to a mean and the uncertainty of that mean. A unit with a small probability of receiving the treatment receives a large inverse-probability weight if treated. That can make the unit important to the estimate, but it can also make the estimate sensitive to the outcome observed for that unit. A variance calculation should reflect both the weights and the outcome residuals; it should not treat the weighted observations as an ordinary unweighted sample.

This tutorial considers the normalized inverse probability weighted mean for the treated group. It derives a first-order sampling-variance estimator under known treatment probabilities and independent treatment assignments, then applies the formula to hypothetical data. The standard error quantifies sampling variability under those assumptions; it does not by itself resolve confounding, poor overlap, or uncertainty from estimating the propensity scores.

Estimator and Assumptions

Definition: For unit $i$, let $A_i$ equal 1 if the unit is treated and 0 otherwise, let $Y_i$ be the observed outcome, and let $e_i=P(A_i=1\mid X_i)$ be the known treatment probability given covariates $X_i$. The normalized inverse probability weighted mean for the treated arm is $$ \hat{\mu}_1= \frac{\sum_{i=1}^{n} A_iY_i/e_i} {\sum_{i=1}^{n} A_i/e_i}. $$ It is defined only when $\sum_{i=1}^{n} A_i/e_i>0$, which is equivalent to observing at least one treated unit.

The target can be described using potential outcomes: $Y_i(1)$ is the outcome unit $i$ would have under treatment. In a fixed finite sample, the target is $\mu_1=N^{-1}\sum_{i=1}^{N}Y_i(1)$. For the variance derivation below, treat the covariates and potential outcomes as fixed and let treatment assignments vary. Assume the $A_i$ are independent Bernoulli variables with probabilities $e_i$, and that $0<e_i\leq1$. The observed outcome satisfies $Y_i=Y_i(1)$ when $A_i=1$. These assumptions make the treatment probability model explicit; causal interpretation additionally requires the substantive conditions that justify using the treatment probabilities to represent the target population.

The estimator is a ratio, not a simple average of observed treated outcomes. Its numerator weights each treated outcome by $1/e_i$, and its denominator normalizes by the total treated weight. Thus a large weight affects the outcome total and the normalization together. This normalization often stabilizes the estimate relative to the unnormalized Horvitz–Thompson mean, but the ratio is not exactly unbiased in finite samples.

Deriving the First-Order Variance Estimator

Write $D=\sum_i A_i/e_i$ for the observed total treated weight. The ratio estimate satisfies the estimating equation $\sum_i A_i(Y_i-\hat{\mu}_1)/e_i=0$. To see how uncertainty enters, first replace $\hat{\mu}_1$ in this equation by the target $\mu_1$. The $i$th contribution is $A_i\{Y_i(1)-\mu_1\}/e_i$. Conditional on the fixed potential outcomes and covariates, its expectation is $Y_i(1)-\mu_1$, and its variance is $\{(1-e_i)/e_i\}\{Y_i(1)-\mu_1\}^2$, because $\operatorname{Var}(A_i)=e_i(1-e_i)$.

A first-order ratio approximation divides the variance of the summed estimating-equation contributions by the square of the expected denominator. In the finite-population target, $\sum_i E(A_i/e_i)=N$. In practice, the unknown residuals are replaced by the observed treated residuals $Y_i-\hat{\mu}_1$, and the unknown variance contribution for each unit is estimated using its observed treatment indicator. This yields the familiar sandwich-style estimator:

\widehat{\operatorname{Var}}(\hat{\mu}_1) = \frac{\displaystyle\sum_{i=1}^{n} A_i\frac{1-e_i}{e_i^2}(Y_i-\hat{\mu}_1)^2} {\displaystyle\left(\sum_{i=1}^{n}A_i/e_i\right)^2}, \qquad \widehat{SE}(\hat{\mu}_1) = \sqrt{\widehat{\operatorname{Var}}(\hat{\mu}_1)}.

The factor in the numerator is $A_i(1-e_i)/e_i^2$, not merely the squared weight $A_i/e_i^2$. Its expectation over treatment assignment is $(1-e_i)/e_i$, the variance factor from the derivation. The denominator uses the square of the observed total weight because it estimates the scale of the ratio’s denominator. This is a first-order variance estimate, not an exact finite-sample variance for every possible treatment assignment.

For the control-arm mean, the corresponding formula follows by using control weights $1/(1-e_i)$:

\hat{\mu}_0= \frac{\sum_{i=1}^{n}(1-A_i)Y_i/(1-e_i)} {\sum_{i=1}^{n}(1-A_i)/(1-e_i)}, \qquad \widehat{\operatorname{Var}}(\hat{\mu}_0)= \frac{\displaystyle\sum_{i=1}^{n}(1-A_i) \frac{e_i}{(1-e_i)^2}(Y_i-\hat{\mu}_0)^2} {\displaystyle\left(\sum_{i=1}^{n}(1-A_i)/(1-e_i)\right)^2}.

Worked Example: Variance of a Treated-Group Mean

Suppose a hypothetical sample has four units. Their known treatment probabilities are $0.50, 0.60, 0.80,$ and $0.40$, and their treatment indicators are $1,0,1,1$. The observed outcomes for the treated units are respectively 10, 8, and 14 units; the outcome for the untreated unit is not used in this treated-arm mean. Calculate the estimate and its variance.

The treated weights are $1/0.50=2$, $1/0.80=1.25$, and $1/0.40=2.5$. Their weighted outcome total is $2(10)+1.25(8)+2.5(14)=65$, and their total weight is $2+1.25+2.5=5.75$. Therefore $\hat{\mu}_1=65/5.75=260/23\approx11.30$ outcome units.

Use the exact mean $260/23$ to calculate the residual contributions. For the first treated unit, the contribution is $(0.50/0.50^2)(10-260/23)^2\approx3.40$. For the third unit it is $(0.20/0.80^2)(8-260/23)^2\approx3.41$. For the fourth unit it is $(0.60/0.40^2)(14-260/23)^2\approx27.25$. The untreated unit contributes zero because $A_i=0$.

The sum of the numerator contributions is approximately $34.06$, and the squared total weight is $5.75^2=33.0625$. Thus $\widehat{\operatorname{Var}}(\hat{\mu}_1)=34.0643/33.0625\approx1.030$ outcome-units squared, and $\widehat{SE}(\hat{\mu}_1)=\sqrt{1.0303}\approx1.015$ outcome units. The estimate is a treated-arm mean of about 11.30, with a first-order estimated standard error of about 1.02 under the stated assumptions.

How Weights and Outcomes Shape Uncertainty

Each treated unit’s numerator contribution is its squared residual from the weighted mean, multiplied by $(1-e_i)/e_i^2$. A unit has little contribution if its outcome is close to the mean, even if its weight is moderately large. Conversely, a treated unit with a small $e_i$ can contribute substantially when its outcome differs from the mean. The denominator also matters: a larger total treated weight reduces the variance estimate for a fixed numerator.

Worked Example: A Low-Probability Treated Unit

In a hypothetical three-unit sample, two units are treated and one is untreated. The treated units have $(e,Y)=(0.20,20)$ and $(0.80,10)$. Find the weighted mean and variance estimate; the third unit’s values do not enter this treated-arm calculation.

The weights are 5 and 1.25. Therefore $\hat{\mu}_1=(5(20)+1.25(10))/(5+1.25)=112.5/6.25=18$ outcome units. The two variance contributions are $(0.80/0.20^2)(20-18)^2=20(4)=80$ and $(0.20/0.80^2)(10-18)^2=0.3125(64)=20$. The numerator is 100 and the squared total weight is $6.25^2=39.0625$, so the estimated variance is $100/39.0625=2.56$ outcome-units squared and the estimated standard error is $1.60$ outcome units.

The low-probability treated unit supplies 80 of the 100 numerator units. This illustrates why an estimate can be highly sensitive to a few observations: inverse weighting corrects for rarity by giving those observations large weights, but large weights can also amplify sampling variability.

Worked Example: Applying the Formula to the Control Arm

Suppose two hypothetical units are untreated, with treatment probabilities $0.25$ and $0.75$, and outcomes 6 and 10. Their control weights are $1/0.75=4/3$ and $1/0.25=4$. The weighted control mean is $\hat{\mu}_0=((4/3)(6)+4(10))/((4/3)+4)=48/(16/3)=9$ outcome units.

The first control’s variance contribution is $(0.25/0.75^2)(6-9)^2=(4/9)(9)=4$. The second’s contribution is $(0.75/0.25^2)(10-9)^2=12$. The contribution total is 16 and the squared total control weight is $(16/3)^2=256/9$. Hence the estimated variance is $16/(256/9)=0.5625$ outcome-units squared, and the estimated standard error is $0.75$ outcome units.

Diagnostics, Alternatives, and Limitations

Before interpreting the standard error, inspect the ingredients of its formula. Review the propensity scores and the largest weights, identify whether a small number of observations dominate the total weight, and compare the corresponding outcomes with the weighted mean. A large variance contribution can result from an extreme weight, a large outcome residual, or both. Also check that both the treatment and control denominators are nonzero if both arm means are needed.

If no treated units are observed, then $\sum_i A_i/e_i=0$, so $\hat{\mu}_1$ and its variance estimate are undefined. When every $e_i<1$, this event has positive probability under independent assignments, even if all probabilities are positive. Do not replace the undefined estimate with zero or report a standard error as if a mean had been observed. State that the estimator cannot be computed for that sample. The formula above is a first-order approximation for samples where the arm is represented; it is not an exact variance conditional on having at least one treated unit. If an empty arm or a few dominant weights are plausible, normal-based inference may be unreliable.

The normalized estimator can be compared with the unnormalized Horvitz–Thompson estimator $n^{-1}\sum_i A_iY_i/e_i$, which targets a finite-population mean and is unbiased under the known-probability assignment model. Its design-based variance estimator uses outcome levels rather than residuals from a fitted weighted mean. The normalized estimator instead divides by the observed total weight; its variance formula consequently uses residuals and the squared total weight. Neither calculation makes extreme weights harmless. Also, treating the weighted outcomes as an ordinary sample and computing their usual unweighted sample variance ignores the assignment probabilities and does not give the variance derived here.

Finally, this calculation assumes the $e_i$ are known. In applications they are often estimated, and the displayed variance does not account for that estimation step. It also relies on independent assignments conditional on the covariates and potential outcomes; dependence, such as clustering, requires a variance method that reflects that sampling structure. The estimate quantifies uncertainty under its stated model, not bias from unmeasured confounding or a lack of support.

Common Mistakes

Dropping the treatment indicator. Only treated units contribute to the treated-arm variance numerator; each contribution includes $A_i$.
Using the wrong propensity factor. The treated contribution is $A_i(1-e_i)/e_i^2$, paired with the squared residual. The $1-e_i$ term is part of the assignment-variance calculation.
Using outcome levels instead of residuals. For the normalized mean, the residual is $Y_i-\hat{\mu}_1$, not $Y_i$.
Assuming the ratio always exists. Positive treatment probabilities do not guarantee a treated observation in every finite sample. If the denominator is zero, the estimate is undefined.
Calling the standard error a causal guarantee. The standard error describes sampling variability under assumptions; it does not establish that the weighted comparison is free from bias.

Key takeaway: For the normalized treated-arm IPW mean, estimate variance by summing treated-unit contributions $A_i(1-e_i)e_i^{-2}(Y_i-\hat{\mu}_1)^2$, then dividing by the squared total treated weight. Large weights and large residuals both increase uncertainty.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why is the treated-arm mean undefined if no units are treated, even when every propensity score is positive?
A treated unit has $e_i=0.50$ and residual $Y_i-\hat{\mu}_1=2$. What is its numerator contribution to the variance estimate?
Why does a large inverse probability weight not necessarily imply a large variance contribution?
In one sentence, distinguish the normalized IPW mean from the unnormalized Horvitz–Thompson mean.
What does the displayed standard error omit when the propensity scores have been estimated from the data?

Solutions: (1) With no treated units, every term in the total treated weight is zero, so the ratio has denominator zero and is undefined. (2) The contribution is $(1-0.50)/0.50^2\times2^2=0.50/0.25\times4=8$. (3) Its contribution also depends on the squared residual; an outcome exactly equal to the weighted mean has zero residual and contributes zero. (4) The normalized mean divides the weighted outcome total by the observed total weight, whereas the Horvitz–Thompson mean divides the weighted outcome total by the sample size. (5) It omits uncertainty from estimating the propensity scores.