From Residual Imbalance to Weighted Effect Bias

Why Residual Imbalance Matters Through the Outcome

A misspecified treatment model can leave covariates imbalanced after weighting, but imbalance alone does not determine the direction or size of an effect error. The relevant question is how the weighting changes the covariate distribution in relation to the conditional outcome mean. If the outcome mean is constant across the residual imbalance, the imbalance does not shift the weighted mean. If it varies with the covariates, the same imbalance can move the estimated effect.

The connection can be stated exactly for a point-treatment setting. The result is a population calculation: it describes the limit targeted by normalized inverse-probability weighting under the stated assumptions, not every finite-sample realization.

Setup and Exact Bias Identity

Let \(A\in\{0,1\}\) be treatment, \(L\) a vector of baseline covariates, and \(Y\) the observed outcome. Write \(Y^a\) for the potential outcome under treatment \(a\), and define the true propensity probabilities \(e_a(l)=P(A=a\mid L=l)\). Let \(q_a(l)\) be the fitted treatment probabilities used to construct weights, with \(q_0(l)+q_1(l)=1\) and \(q_a(l)>0\) wherever \(e_a(l)>0\). The outcome regression is \(m_a(l)=E[Y\mid A=a,L=l]\).

For identification of the average treatment effect, assume consistency, conditional exchangeability \(Y^a\perp A\mid L\), and positivity \(e_a(L)>0\) almost surely for \(a=0,1\). Also assume the relevant expectations are finite. These conditions give \(m_a(l)=E[Y^a\mid L=l]\), so the target means are \(\mu_a=E[m_a(L)]=E[Y^a]\), and the target effect is \(\tau=\mu_1-\mu_0\).

The population counterpart of the normalized weighted mean in arm \(a\) is

Define the ratio \(r_a(L)=e_a(L)/q_a(L)\). Require \(0<E[r_a(L)]<\infty\) and \(E[r_a(L)|m_a(L)|]<\infty\). Conditional expectation gives the following identity.

Proof. Given \(L\), \(P(A=a\mid L)=e_a(L)\), and \(E[I(A=a)Y\mid L]=e_a(L)m_a(L)\). Therefore

Their ratio is the stated expression for \(\mu_a^{\,q}\). Since \(E[r_am_a]=E[r_a]E[m_a]+\operatorname{Cov}(r_a,m_a)\), subtracting \(\mu_a=E[m_a]\) gives the arm-specific bias identity. Subtracting the two target arm means from the two weighted arm means gives the effect identity. \(\square\)

The same calculation describes covariate imbalance. If \(F\) is the population distribution of \(L\), the weighted distribution in arm \(a\) has density relative to \(F\)

Thus \(\mu_a^{\,q}=\int m_a(l)\,dG_a^q(l)\), while \(\mu_a=\int m_a(l)\,dF(l)\). The weighted mean shifts when this change in distribution places different weight on covariate values with different outcome means. This also shows why a misspecified treatment model does not guarantee imbalance: if \(r_a(l)\) is constant over \(l\), then \(G_a^q=F\).

Continuous Covariate and a Bias Bound

The identity applies to continuous covariates as well. It can also be bounded in terms of how much the ratio varies and how strongly it covaries with the outcome regression.

Derivation. The bias identity and the Cauchy–Schwarz inequality give \[ |\mu_a^{\,q}-\mu_a| = \frac{|\operatorname{Cov}(r_a,m_a)|}{E[r_a]} \leq \frac{\sqrt{\operatorname{Var}(r_a)\operatorname{Var}(m_a)}}{E[r_a]}. \] All quantities are finite by assumption, and the positive denominator permits division. \(\square\) This bound is not generally sharp: it controls the magnitude but does not reveal the sign, and the two arm-specific biases can partially cancel or reinforce one another in the effect contrast.

Longitudinal Interpretation

For a longitudinal regime \(\bar a=(a_0,\ldots,a_K)\), let \(H_t\) denote the observed history immediately before treatment at time \(t\). Write \(e_t(a_t\mid H_t)\) for the true conditional treatment probability and \(q_t(a_t\mid H_t)\) for its fitted counterpart. Under the usual sequential identification conditions, the ratio comparing the true and fitted treatment factors along a regime is

In the population weighted regime sample, the regime-history distribution is reweighted by \(R\) and then normalized. Accordingly, a history-specific outcome mean that is higher in histories receiving relatively large \(R\) contributes more to the weighted mean. This is the longitudinal counterpart of the point-treatment covariance relationship. It does not imply that fitted probabilities being wrong must change the normalized history distribution: if \(R\) is constant over histories under the regime distribution, normalization removes that constant. Nor does history imbalance alone determine outcome bias; its relationship to the conditional outcome mean matters.

This interpretation requires the regime to be supported by the observed treatment process, valid sequential exchangeability and consistency, and finite weighted expectations. With many treatment times, small ratio errors can multiply, and histories with very large or very small product ratios can make estimates unstable. The point-treatment identity is a guide to the mechanism, not a substitute for checking longitudinal support and weight behavior.

Diagnostics, Alternatives, and Limitations

Post-weighting balance diagnostics examine whether the weighted covariate distribution resembles the intended target. They are useful for detecting some forms of residual imbalance, but they do not directly measure the covariance between the weight ratio and the outcome regression. Balance on a few means can miss distributional differences that matter for nonlinear outcome relationships; conversely, measurable imbalance may have little effect when the outcome mean changes little across the imbalanced values. Outcome-blind diagnostics therefore cannot establish that weighted effect bias is absent.

The identity also clarifies the roles of alternative approaches. Inverse-probability weighting relies on the treatment model to construct a weighted population and is vulnerable when its ratios remain associated with outcome-relevant covariates. Outcome-regression standardization instead relies on modeling the conditional outcome mean and averaging predictions over the target covariate distribution; its vulnerability is outcome-model misspecification rather than residual imbalance induced by treatment weights. These approaches make different modeling commitments, and neither is automatically protected from all misspecification.

A frequent mistake is to argue, “the weighted covariates are imbalanced, so the treatment effect must be biased by that amount.” This is wrong: an imbalance is a change in a covariate distribution, not an outcome contrast. The correction is to consider how the change acts on \(m_a(L)\), arm by arm, as the covariance identity does. A second mistake is to infer that an incorrect treatment model necessarily creates imbalance. Constant true-to-fitted ratios within an arm produce no normalized distribution shift, as the third example demonstrates.