Variance Estimation for Inverse-Probability-Weighted Effects

Why Variance Estimation Must Account for Estimated Weights

Inverse-probability weighting makes a treatment-effect estimate depend on both the observed outcomes and the propensity scores used to weight them. When those scores are fitted from the same data, the resulting estimate has uncertainty from both sources. A variance calculation that treats the fitted scores as known generally omits one part of that uncertainty.

This tutorial develops the adjustment for the average treatment effect (ATE). The same estimating-equation approach applies to other inverse-probability-weighted effects and to normalized weighted estimators, but their estimating equations and derivatives must match the estimator actually used. The aim is not to claim that estimating scores must always increase the standard error: the adjustment can increase or decrease it, depending on how the propensity-score fit responds to the data.

Under these identification assumptions, the ATE is \(\psi=E[Y(1)-Y(0)]\). With \(e_i(\beta)=e(X_i;\beta)\), the inverse-probability-weighted estimator considered here is

This is the unnormalized, or Horvitz–Thompson, form of the IPW estimator. Define its individual contribution at a generic value of \(\beta\) as \(m_i(\beta)\), the expression inside braces. If the true propensity scores were known, the influence function would be \(m_i-\psi\). Estimating \(\beta\) changes that influence function.

From a Fitted Propensity Score to an Influence Function

Suppose \(\hat\beta\) solves a propensity-score estimating equation \(n^{-1}\sum_i s_i(\hat\beta)=0\), where \(s_i\) is the score contribution. For a logistic model, for example, \(e_i(\beta)=\{1+\exp(-X_i^\mathsf{T}\beta)\}^{-1}\) and \(s_i(\beta)=X_i\{A_i-e_i(\beta)\}\). Define the expected information matrix \(I=E[e_i(1-e_i)X_iX_i^\mathsf{T}]\). Under the stated regularity conditions, the first-order influence function for \(\hat\beta\) is \(I^{-1}s_i\).

Let \(G=E[\partial m_i(\beta)/\partial\beta]\), evaluated at the true parameter. A first-order expansion of the weighted estimate around the true propensity parameter gives the influence function

The second term is the correction for estimating the propensity-score parameter. In a fitted analysis, its empirical counterpart uses the average derivative \(\hat G\), the fitted information matrix \(\hat I\), and fitted score contributions \(s_i(\hat\beta)\). The estimated influence values are then

Derivation Through Estimating Equations

The correction follows directly from a joint expansion. Stack the effect and propensity-score equations: the first equation sets the sample average of \(m_i(\beta)-\psi\) to zero, and the second sets the average score to zero. For a logistic propensity model, the expected derivative of the score with respect to \(\beta\) is \(-I\). The expected derivative of the first equation with respect to \(\beta\) is \(G^\mathsf{T}\). Thus the Jacobian of the stacked equations has the block form below.

For one observation, the stacked equation contribution is \((m_i-\psi,\ s_i^\mathsf{T})^\mathsf{T}\). Solving the linearized second equation gives the parameter influence function \(I^{-1}s_i\). Substituting it into the first linearized equation gives

Here \(\doteq\) denotes equality to first order, with terms that are negligible at the \(\sqrt n\) scale omitted. This is also the sandwich calculation: the covariance of the stacked equation contributions is placed between the inverse Jacobian and its transpose. Extracting the effect component yields \(E[\phi_i^2]/n\), estimated by the empirical influence-value variance divided by \(n\).

Normalized Estimators and the Estimator Actually Used

The variance calculation must correspond to the point estimator. For example, a Hájek-normalized weighted mean is a ratio, not the unnormalized weighted mean used above. For a generic ratio \(\hat\mu=\bar U/\bar V\), with \(E[V]\ne0\), its first-order contribution when \(U\) and \(V\) are known functions of the observation is \((U_i-\mu V_i)/E[V]\). This follows by expanding the ratio around \(E[U]/E[V]\). If either \(U\) or \(V\) also depends on \(\hat\beta\), its derivative with respect to \(\beta\) contributes to the nuisance-parameter adjustment as well.

In practice, one can stack the estimating equations for the exact normalized effect and propensity-score model, calculate their Jacobian and empirical equation contributions, and form the sandwich variance. Alternatively, derive the corresponding influence function by expansion and estimate its second moment. Using the unnormalized ATE influence function for a normalized estimator, or using the wrong target population after trimming, does not give valid inference for the estimator and target actually reported.

Common Pitfalls and Interpretation

• Holding fitted scores fixed. Treating \(\hat\beta\) as known leaves out \(G^\mathsf{T}I^{-1}s_i\). Use the fitted-score correction, unless propensity scores truly were fixed externally and the variance target explicitly conditions on them.
• Using the wrong derivative or sign. The correction follows from the derivative of the weighted contribution and the influence function of the fitted parameter. Check the estimating-equation convention: for logistic maximum likelihood, the expected score derivative is \(-I\), which yields \(\mathrm{IF}_{\hat\beta}=I^{-1}s_i\).
• Assuming the adjustment must inflate uncertainty. The corrected values can be more or less variable than the raw values. Their cross-product with the correction determines the direction.
• Confusing a standard error with a causal assumption. A sandwich standard error quantifies sampling uncertainty under the model and regularity conditions; it does not establish exchangeability, positivity, correct specification, or causal identification.

Under the stated conditions, the estimator is asymptotically normal, so a large-sample interval can be formed as \(\hat\psi\pm z_{1-\alpha/2}SE(\hat\psi)\). Its interpretation is repeated-sampling coverage for the specified target under the assumed model, not a probability statement that the fixed causal effect lies in this particular interval. Very small propensity scores can also produce unstable contributions and unreliable normal approximations; overlap diagnostics and the target-population decision remain substantively important.