Treatment Prediction Versus Confounder Control

Why Treatment Prediction Is Not the Causal Objective

A propensity model estimates how treatment assignment varies with measured characteristics. It is tempting to judge such a model by how accurately it predicts who was treated. But accurate classification answers a prediction question, not the causal question: have treated and untreated people been made comparable with respect to causes of the outcome? A model can predict treatment very well because it captures a powerful treatment-assignment mechanism, yet leave an important confounder uncontrolled. Conversely, a measured confounder may add little to treatment prediction while substantially affecting the outcome.

Let \(A\) be a binary treatment, \(Y\) the observed outcome, and \(Y(1)\) and \(Y(0)\) the potential outcomes under treatment and control. Let \(X\) denote measured pretreatment covariates, and define the propensity score \(e(X)=P(A=1\mid X)\). For an average treatment effect, a central target is \(E[Y(1)-Y(0)]\). Estimating it from observed outcomes requires assumptions about treatment assignment and the covariates used to adjust for it.

For identification by adjustment, the covariates must be measured before treatment, treatment assignment must be conditionally exchangeable given those covariates, and both treatment options must be possible at covariate values relevant to the target population. In potential-outcome notation, these conditions include consistency, \(Y=AY(1)+(1-A)Y(0)\); conditional exchangeability, \((Y(1),Y(0))\perp A\mid X\); and positivity, \(0<P(A=1\mid X)<1\). Consistency also requires a sufficiently well-defined treatment, and the usual individual-level interpretation assumes no interference between units. The propensity model must estimate the relevant assignment probabilities adequately for the chosen adjustment procedure. Prediction accuracy is not a substitute for any of these assumptions.

The Balancing Property—and Its Boundary

The balancing property has a short derivation. Fix a score value \(e(X)=s\), where \(0<s<1\), and any set of covariate values \(B\). The probability of both being treated and having \(X\) in \(B\), conditional on the score, is \(sP(X\in B\mid e(X)=s)\). Dividing by the probability of treatment at that score gives the conditional distribution among treated people:

For controls, the same argument uses \(1-s\), so their distribution of \(X\) also equals \(P(X\in B\mid e(X)=s)\). Thus treatment status is independent of the measured \(X\) within an exact propensity-score stratum. This is a distributional statement, not just a statement about means.

The boundary is crucial: the result concerns the covariates used to define the score. If \(X\) contains only a strong treatment predictor \(Z\) and omits a confounder \(U\), balancing \(Z\) does not balance \(U\). A high area under the receiver operating characteristic curve, classification accuracy, or treatment-model likelihood does not establish conditional exchangeability. The introductory material on propensity scores establishes their role in adjustment; here the issue is why optimizing treatment prediction alone may undermine that role.

Strong Predictors, Poor Overlap, and Weight Instability

Even when a strong treatment predictor is not a confounder, including it in a propensity model can have a practical cost. If it drives estimated scores close to zero or one, few comparable treated and untreated people remain in parts of the covariate space. In inverse probability weighting for the average treatment effect, treated observations receive weight \(1/e(X)\) and controls receive weight \(1/[1-e(X)]\). Scores near the boundaries can consequently produce large weights, high variance, and a target estimate dominated by a few observations. Under ideal population conditions, including a valid instrument alongside all confounders does not by itself invalidate identification; the concern is often loss of precision, finite-sample instability, or amplified bias when confounding remains unresolved.

One useful summary is the weighted effective sample size, \(ESS=(\sum_i w_i)^2/\sum_i w_i^2\). It describes how much information the weights retain relative to an equally weighted sample; it is not a replacement for checking balance or overlap.

In practice, inspect the estimated-score distributions by treatment group, the number and characteristics of observations near the boundaries, weight summaries, and effective sample sizes. After matching, weighting, or subclassification, assess covariate balance directly, commonly with standardized mean differences and distributional comparisons. Check important confounders and prognostic covariates rather than relying on a single overall balance summary. Large residual imbalances, extreme weights, or very small effective sample sizes are warnings that the design may not support stable adjustment for the intended target population.

Weak Treatment Predictors Can Be Important Confounders

Treatment prediction also provides a poor filter for deciding whether a covariate matters causally. A variable can have a weak association with treatment but a strong association with the outcome. If it is a common cause of treatment and outcome, even a modest treatment association can generate meaningful confounding. The direction and importance of confounding depend on the joint relationship with treatment and outcome, not on treatment-model performance alone.

This distinction informs model assessment: treatment discrimination and causal balance are different objectives. A treatment model can be compared using predictive measures, but the design should be assessed through overlap and covariate balance. Outcome prognostic information can help prioritize which imbalances are consequential, while a variable's causal role still depends on the assignment and outcome structure. Adding variables indiscriminately is not a solution: covariates should be pretreatment, and adjustment must respect the causal structure.

Choosing and Diagnosing an Adjustment Strategy

Propensity-score matching, weighting, and subclassification use different implementations of the same broad idea: compare treatment groups after adjustment for measured pretreatment covariates. None repairs an omitted confounder merely by achieving excellent treatment prediction. Outcome regression models the outcome conditional on covariates instead; it may be more efficient in some settings, but it also relies on adequate covariate measurement and model specification. A carefully designed analysis can use propensity adjustment, outcome adjustment, or both, while explicitly stating the target population and assumptions.

Instrumental-variable analysis addresses a different identification strategy. A variable that predicts treatment but is independent of outcome causes and affects the outcome only through treatment may serve as an instrument under additional assumptions. Its treatment-prediction strength is relevant there, but it does not make it a confounder-control variable. Simply adding an instrument to a propensity model does not convert propensity adjustment into an instrumental-variable analysis. When residual confounding is present, conditioning on a strong instrument can, in some settings, amplify bias; it can also worsen overlap. Neither effect is automatic in every design, so the causal structure and observed diagnostics matter.

The limits of diagnostics should also be explicit. Balance checks evaluate measured covariates, not unmeasured confounding. Overlap plots cannot prove that positivity holds for all relevant covariate patterns, and acceptable effective sample size does not establish correct model specification. When estimates rely on trimming or restricting the sample to improve overlap, the target population may change and should be described. Robust causal interpretation requires substantive knowledge alongside statistical checks.