Clipping Bias in a Binary-Treatment Mean Estimate

Why clipping changes a mean estimate

Inverse probability weights compensate for differences in treatment probabilities across covariate strata. Capping a large weight reduces the influence of observations with small treatment probabilities. That can improve numerical stability, but it also changes how much each stratum contributes to the weighted mean. For a treatment contrast, the change in one arm need not match the change in the other.

The goal here is to calculate that change directly in a finite-stratum setting. This makes the source and direction of the bias visible: the relevant quantities are the treatment probabilities, the cap, and the outcome means in each stratum.

Setup and definition

Let \(A\in\{0,1\}\) be a binary treatment and let \(X\) take values in a finite set of strata. Write \(q_x=P(X=x)\), \(p_1(x)=P(A=1\mid X=x)\), and \(p_0(x)=1-p_1(x)\). Let \(Y^a\) be the potential outcome under treatment \(a\), and let \(Y\) be the observed outcome.

Assume consistency, so \(Y=Y^A\); conditional exchangeability, \((Y^0,Y^1)\perp A\mid X\); and positivity, \(0<p_a(x)<1\) whenever \(q_x>0\). Define \(\mu_a(x)=E[Y^a\mid X=x]\). Under these assumptions, \(\mu_a(x)=E[Y\mid A=a,X=x]\), and the causal mean under treatment \(a\) is \(\psi_a=E[Y^a]=\sum_x q_x\mu_a(x)\). The causal mean contrast is \(\Delta=\psi_1-\psi_0\).

Suppose the probabilities \(p_a(x)\) are known, or for now are treated as the true probabilities. For a fixed cap \(c\geq 1\), clip the inverse probability weight for arm \(a\) from above at \(c\). The unnormalized, or Horvitz–Thompson, clipped mean estimate is the sample average of \(I(A=a)Y\min\{1/p_a(X),c\}\).

Exact bias for the unnormalized clipped contrast

The retention factor equals one when the arm’s inverse probability weight is not capped. If \(1/p_a(x)>c\), then \(g_a(x)=c\,p_a(x)<1\). Thus clipping removes expected contribution precisely in strata where the arm is relatively unlikely.

To prove the result, condition on \(X=x\). By conditional exchangeability and consistency, \(E[Y\mid A=a,X=x]=\mu_a(x)\). Therefore,

Averaging over the finite strata gives the stated expectation. Subtracting \(\psi_a=\sum_x q_x\mu_a(x)\) gives \(B_a(c)\). Subtracting the two arm expectations and then subtracting \(\Delta\) gives \(B_\Delta(c)=B_1(c)-B_0(c)\), which is the displayed expression. \(\square\)

This is an exact expectation statement for the unnormalized estimate, not an approximation that requires a large sample. In particular, its contrast bias can have either sign. When outcomes are nonnegative, clipping lowers each arm’s unnormalized mean estimate, but it lowers the two arms by potentially different amounts. Subtracting those estimates can therefore raise or lower the contrast.

Why the bias need not move a contrast toward zero

A common intuition is that clipping must attenuate the estimated treatment effect. The calculation above does not justify that claim: clipping changes the representation of each arm’s outcome distribution, not just the size of the contrast. Even a constant treatment effect across strata does not guarantee attenuation when the baseline outcome means vary.

Normalized weighted means target a different limit

The unnormalized estimate has a simple exact expectation, but many analyses use a normalized, or Hájek, weighted mean: the weighted outcome sum divided by the sum of the weights in that arm. A ratio’s finite-sample expectation is generally not the ratio of its expectations. Its large-sample limit, however, has a direct expression under simple regularity conditions.

For completeness, write the numerator and denominator of the sample ratio as sample averages \(N_n\) and \(D_n\). Each summand is bounded because \(Y\) is bounded and the clipped weight is at most \(c\). The law of large numbers gives \(N_n\xrightarrow{p}\sum_x q_xg_a(x)\mu_a(x)\) and \(D_n\xrightarrow{p}\sum_x q_xg_a(x)>0\). The continuous mapping theorem for division at a nonzero denominator then gives \(N_n/D_n\xrightarrow{p}\psi_a^{H}(c)\). The positive denominator also implies \(P(D_n=0)\to0\), so the ratio can be defined arbitrarily on the event that no arm-\(a\) weight is observed without changing this probability limit. \(\square\)

In contrast, the unnormalized clipped mean’s probability limit is its expectation, \(\sum_x q_xg_a(x)\mu_a(x)\). Normalization divides by the retained total mass; it does not generally restore the original target, because strata with different retention factors still receive different relative weights.

Diagnostics, alternatives, and limitations

A useful diagnostic is to calculate \(g_a(x)=\min\{1,cp_a(x)\}\) for each arm and relevant covariate stratum. Values below one identify contributions reduced by the cap; values near zero identify strata whose arm-specific outcomes contribute very little to the clipped estimate. Also report how the contrast changes across a prespecified range of caps. A material change signals that the estimate depends on the clipping rule, although stability across caps does not prove that clipping bias is absent.

The main comparison is with the unclipped inverse-probability-weighted mean: without a cap, \(g_a(x)=1\) in every stratum, and the exact expectation equals \(\psi_a\). Clipping trades that property for bounded weights. Deleting observations above a cutoff is a different rule: it can remove observations rather than retaining them at a capped weight, and it need not have the same expectation. The earlier material on How Truncation Changes the Estimating Equation explains why changing weights also changes the estimating equation.

The formulas above assume known propensity scores. If probabilities are estimated, the clipped weights are random and the exact expectation calculation conditional on the true \(p_a(x)\) no longer describes the full estimator automatically. The formulas also do not imply that a single cap is appropriate in every setting. They quantify the bias for the stated rule and target; choosing a cap requires considering the consequences of changing that target as well as the practical reasons for limiting extreme weights.