Positivity, Extreme Weights, and Information

Why Positivity Matters for Weighting

Inverse-probability weighting uses each observation to represent people with similar measured covariates who received the other treatment. When a person’s treatment is highly predictable from those covariates, that representation can require a large weight. A handful of such observations can then dominate an estimate, making it sensitive to individual outcomes and reducing the amount of useful information in the weighted sample.

Let \(A\) be a binary treatment, with \(A=1\) for treated and \(A=0\) for control, and let \(X\) denote measured pretreatment covariates. The propensity score is \(e(X)=P(A=1\mid X)\). Let \(Y(1)\) and \(Y(0)\) be the potential outcomes under treatment and control, and define the average treatment effect (ATE) as \(\tau=E[Y(1)-Y(0)]\). In the examples below, all data are hypothetical.

Positivity works alongside consistency and conditional exchangeability, \(Y(1),Y(0)\perp A\mid X\), for identifying the ATE from observed data. Positivity alone does not remove confounding. Conversely, even if positivity holds strictly at every covariate value, probabilities can be so close to zero or one that weighting is practically unstable in a finite sample.

From Small Probabilities to Large Weights

For the ATE, the inverse-probability weight is \(1/e(X)\) for a treated observation and \(1/[1-e(X)]\) for a control observation. The Horvitz–Thompson form of the IPW estimator is

When \(e(X_i)\) is small, a treated person receives a large weight \(1/e(X_i)\). When \(e(X_i)\) is near one, a control person receives a large weight \(1/[1-e(X_i)]\). The key is the probability of the treatment actually received: a treated person with \(e=.02\) has weight \(50\), while a control person with \(e=.98\) also has weight \(50\). In either case, that person’s outcome can exert substantial influence on the estimate.

A direct justification for why large weights matter comes from their second moments. Suppose the propensity score is the true \(e(X)\), conditional exchangeability holds, and potential outcomes have finite second moments. Write the summand in the estimator as \(\psi=AY/e(X)-(1-A)Y/[1-e(X)]\). Conditional on \(X\), the treated and control terms cannot both be nonzero. By consistency and exchangeability,

For example, the treated contribution follows because \(A^2=A\) and \(E[A\mid X]=e(X)\), so \(E[A Y^2/e(X)^2\mid X]=E[Y(1)^2\mid X]/e(X)\). The control contribution follows in the same way using \(1-A\), whose conditional probability is \(1-e(X)\). As \(e(X)\) approaches zero, the first term can grow sharply; as it approaches one, the second can. These second moments help explain why extreme weights can make estimates variable even when all probabilities are technically positive. The outcome magnitudes also matter, so a large weight does not determine the variance by itself.

Weight Concentration and Effective Sample Size

A useful summary of how weights are distributed within an arm is the Kish effective sample size. For nonnegative weights \(w_1,\ldots,w_m\), define \(n_{\mathrm{eff}}=(\sum_j w_j)^2/\sum_j w_j^2\). Equal weights give \(n_{\mathrm{eff}}=m\); concentration in a few weights makes it smaller. The measure is unchanged if all weights are multiplied by the same positive constant.

Calculate this diagnostic separately for treated and control weights: each arm has a different weight formula and may have a different concentration pattern. It can also be informative to inspect the largest weights, upper quantiles, and the share of total weight carried by a small number of observations. A single summary can conceal whether concentration is driven by one extreme case or by a broader tail.

How Extreme Weights Can Move an Estimate

The effect of a large weight depends on both the weight and the observation’s outcome. A normalized, or Hájek, weighted mean in the treated arm is \(\sum_i A_iY_i/e(X_i)\) divided by \(\sum_i A_i/e(X_i)\). The next example shows how one observation can have substantial leverage on that mean.

Diagnostics, Alternatives, and Limitations

Weight diagnostics are most useful when interpreted together. Inspect the propensity-score distributions by treatment arm, the largest arm-specific weights, effective sample sizes, and how much total weight is carried by the most heavily weighted cases. Then check whether the extreme scores correspond to substantively unusual covariate profiles or sparse treatment cells. The earlier material on support and tail diagnostics motivates these checks; here the emphasis is on the information consequences of the resulting weights.

Distinguish a structural positivity failure from practical instability. If a covariate profile in the target population can only receive one treatment, strict positivity fails there, and the ATE is not identified from those data without additional assumptions. If probabilities are positive but very small, positivity may hold mathematically while the finite-sample estimate remains highly variable. Estimated scores close to zero or one flag a concern, but do not prove the true probabilities are exactly zero or one. This tutorial concerns the consequences of weight patterns, not sampling variability in estimated support.

Different approaches address different parts of the problem. Matching seeks comparable treated and control observations, while outcome regression models outcomes conditional on covariates; neither makes unsupported treatment comparisons automatically credible. Trimming or restricting to a supported region can reduce extreme weights, but changes the population to which the estimate applies. No method can recover information about a treatment contrast that the observed design does not support without extra assumptions. The choice should therefore be guided by the target population and the scientific question, not by a preference for smaller weights alone.

• Do not equate a large weight with an error. It reflects the weighting scheme for the target estimand; investigate its implications rather than deleting it automatically.
• Do not treat effective sample size as a variance estimate. It summarizes weight concentration but does not incorporate every feature of outcomes, dependence, or the estimator.
• Do not assume that clipping weights is cost-free. Capping weights can reduce the influence of extreme observations but generally changes the weighting and may introduce bias for the original estimand.
• Do not infer adequate overlap from a moderate average weight. Averages can hide a small number of very large weights or a poorly supported subgroup.