Consequences of Misspecified Treatment Weight Models

Why Treatment Model Misspecification Matters

Inverse-probability weighting is intended to construct a pseudo-population in which measured confounders no longer predict treatment. That purpose depends on the treatment probabilities used to calculate the weights. If the probability model misses important features of treatment assignment, weighting can leave treatment groups different in their confounder distributions. The resulting weighted comparison may therefore retain confounding, even when the outcome model is correctly specified for the intended marginal structural model.

The key issue is not simply whether the weights are large or whether the treatment model predicts treatment well overall. The issue is whether the fitted probabilities reproduce treatment assignment conditional on the measured covariates and histories needed to control confounding. The calculations below make the connection between probability errors, imbalance, and bias explicit.

Setup and the Role of the Treatment Model

Begin with one binary treatment \(A\), measured covariates \(X\), and an outcome \(Y\). Let \(Y^1\) and \(Y^0\) denote the potential outcomes under treatment and no treatment. The causal contrast considered here is the average treatment effect \(E[Y^1-Y^0]\). For this contrast to be identified by treatment weighting, assume consistency, conditional exchangeability \( (Y^1,Y^0)\perp A\mid X\), and positivity: every covariate pattern in the target population has a positive probability of receiving either treatment.

Define the true propensity score \(e(X)=P(A=1\mid X)\), and let \(\hat e(X)\) denote the probability supplied by the fitted treatment model. The unstabilized inverse-probability weight for an individual is \(1/\hat e(X)\) if \(A=1\), and \(1/\{1-\hat e(X)\}\) if \(A=0\). In a weighted comparison, each treatment group’s outcome mean is calculated using its corresponding weights.

The following identity shows why correct conditional probabilities matter. Conditional on \(X\), the expected weighted contribution to the treated group is \(E[A/\hat e(X)\mid X]=e(X)/\hat e(X)\). The corresponding contribution to the untreated group is \(E[(1-A)/\{1-\hat e(X)\}\mid X]=\{1-e(X)\}/\{1-\hat e(X)\}\). If \(\hat e(X)=e(X)\), both expressions equal 1, so weighting restores the same relative representation of each covariate pattern in the two groups. If the probabilities are wrong, those expressions generally vary with \(X\), and the groups need not have the same covariate distribution.

Deriving the Residual Imbalance

For a discrete covariate pattern \(x\) with population proportion \(f(x)\), the treated group’s expected weighted mass at that pattern is proportional to \(f(x)e(x)/\hat e(x)\). The untreated group’s expected weighted mass is proportional to \(f(x)\{1-e(x)\}/\{1-\hat e(x)\}\). These are then normalized separately to give the covariate distributions within the two weighted groups.

When the fitted probabilities are correct, both unnormalized distributions are proportional to \(f(x)\), so normalization makes them equal. Under misspecification, their relative weights across covariate patterns differ. This argument concerns the treatment-weighting step; it does not require the outcome to be continuous or the treatment effect to be constant. The numerical examples use simple outcomes so the consequences can be calculated directly.

Partial Probability Errors Still Leave Bias

A model does not have to omit a covariate entirely to cause consequential imbalance. It may capture part of the variation in treatment probabilities but still get their conditional values wrong. The next calculation holds the population and causal effect fixed while replacing the constant fitted probability with probabilities that are closer to the truth.

Misspecification Across Covariate Patterns

The conditional-probability identity applies to every covariate pattern, not just to a single omitted binary confounder. An overall treatment proportion can agree with its fitted counterpart while conditional probabilities are wrong. Such marginal agreement does not ensure that the weighted groups have matching distributions of prognostic covariates.

Why the Consequences Can Compound Over Time

In longitudinal analyses, treatment weights involve probabilities conditional on treatment and covariate histories. For a binary treatment at visits \(t=0,\ldots,K\), an unstabilized treatment weight for an observed treatment history is a product of inverse conditional probabilities. A stabilized weight uses a numerator probability as well; in either case, the denominator model must represent treatment probabilities conditional on the measured history needed to control confounding.

At a given visit, let \(H_t\) denote the relevant observed history before treatment, and write \(e_t(H_t)=P(A_t=1\mid H_t)\) for the true probability and \(\hat e_t(H_t)\) for the fitted probability. The same conditional calculation applies at that visit: the expected treated contribution after applying the fitted inverse probability is \(e_t(H_t)/\hat e_t(H_t)\), and the untreated contribution is \(\{1-e_t(H_t)\}/\{1-\hat e_t(H_t)\}\). If these ratios vary across histories, the weighted treatment groups can remain imbalanced within the history distribution.

Because a longitudinal weight multiplies contributions from multiple visits, probability errors at more than one visit can interact. For example, a model that misses a nonlinear history effect at one visit can leave imbalance in a time-varying confounder; a later model may then also assign incorrect probabilities across histories affected by that imbalance. This does not mean every small error mechanically creates a large bias. The consequence depends on which histories are misweighted, how strongly their covariates predict the outcome, and how treatment effects vary across the target population.

What Model Checks Can and Cannot Establish

Balance checks examine whether measured covariates and histories are similarly distributed across weighted treatment groups. They address a consequence of treatment model fitting rather than proving that a model is correct. The earlier material on covariate balance and weight-model diagnosis is useful here: balance should be examined for relevant covariates and, where needed, transformations and interactions, not inferred from a single overall fit statistic.

A well-behaved weight distribution is not proof of adequate confounding control. In the first worked example, all weights were identical and finite, yet the weighted groups retained severe risk imbalance. Conversely, extreme weights may indicate limited practical overlap or a poorly estimated model, but avoiding extreme weights alone does not guarantee that the remaining weights remove confounding.

Model changes should be guided by the treatment-assignment process and subject-matter knowledge, with balance and overlap assessed after fitting. If the measured data do not support adequate adjustment for some histories, changing the functional form cannot create information that is absent from the data. Weight truncation also does not correct the underlying probability model: it may trade some variance for additional bias, rather than restore conditional balance.

Common Mistake: Treating a Good Marginal Fit as Conditional Balance

• Wrong reasoning: “The predicted treatment proportion matches the observed proportion, so the treatment weights must have removed confounding.” The four-stratum example shows why this fails: the overall proportions both equal 0.50, while conditional treatment probabilities are wrong and the weighted effect is biased.
• Correction: The required probabilities are conditional on the measured confounders or histories, not merely calibrated in the aggregate. Evaluate whether the weighted groups are balanced on relevant measured features and whether the model represents plausible treatment-assignment structure.
• Another wrong reasoning: “A robust standard error or bootstrap will fix the bias from a misspecified treatment model.” These procedures quantify sampling uncertainty under their respective assumptions; they do not make an incorrect weighting model correct. A narrow interval around a biased estimate can still be misleading.