Avoiding Future Covariates in Treatment Models

Why the Timing of a Predictor Matters

A treatment probability model should describe the chance of receiving treatment using information available when that treatment decision is made. In longitudinal data, this sounds straightforward until measurements and decisions are recorded by visit or date rather than by their actual order. A value labeled “week 4” might have been collected before the week 4 treatment decision, after it, or at an unclear point during the visit. Those distinctions matter: a measurement recorded after treatment cannot be used to represent the information available to assign that treatment.

The issue is not merely whether a variable predicts treatment well. A future measurement may predict an earlier treatment because treatment affects that measurement, because both reflect underlying illness, or because the data contain information from the future. Conditioning on it can change the treatment probabilities used for weighting and undermine the intended comparison.

Define the History at Each Decision

Let \(A_t\) denote treatment at decision time \(t\). Let \(L_t\) denote covariates measured immediately before that decision, and let \(L_{t+1}\) denote measurements taken after \(A_t\) and before the next decision. The observed history immediately before \(A_t\) is

The treatment probability at time \(t\) is the conditional probability of the treatment actually assigned, given that pre-decision history. For binary treatment, for example, the model describes \(P(A_t=1\mid H_t)\) and \(P(A_t=0\mid H_t)\). A measurement belongs in \(H_t\) only if it was available before \(A_t\) was decided.

This rule is relative to the particular decision. \(L_2\) is future information for a decision at \(t=1\), but it can be a valid pre-decision covariate for a later decision at \(t=2\), if it was measured before \(A_2\). Likewise, a value collected during a visit is not automatically valid or invalid: the actual order of measurement and treatment assignment determines its status.

Why Future Measurements Do Not Belong in the Treatment Model

The inverse probability of treatment weight uses probabilities for the treatment assignments along a person's observed history. For a fixed treatment sequence \(\bar a=(a_0,\ldots,a_K)\), its unstabilized regime weight has the form

The denominator represents sequential assignment probabilities given the information available at each decision. Replacing \(H_t\) by a history that includes \(L_{t+1}\) changes the conditional probability being modeled: it becomes a probability of an earlier assignment given later information. That is not the assignment mechanism at time \(t\). In particular, if treatment affects \(L_{t+1}\), conditioning on that measurement can condition on a consequence of the treatment itself.

The following identification result states the role of the correctly timed history. For clarity, take a finite number of decision times and discrete covariates and outcomes; the same argument applies to continuous variables using conditional densities. Assume consistency, sequential exchangeability given each \(H_t\), and positivity: every treatment required by the regime has positive probability at histories that can occur under that regime.

Here, consistency means that a person who follows \(\bar a\) has observed outcome \(Y=Y^{\bar a}\). Sequential exchangeability means that, at each decision, treatment assignment is independent of the relevant future potential outcomes conditional on the observed pre-decision history among people following the regime up to that time. Positivity requires \(P(A_t=a_t\mid H_t)>0\) at histories with positive probability under the regime.

Derivation. Write the observed-data distribution as a product of successive treatment probabilities and conditional distributions of covariates and the outcome. Restricting to the regime \(\bar a\) selects the factors for \(A_t=a_t\). Multiplying by the indicator of following \(\bar a\) and dividing by \(\prod_t P(A_t=a_t\mid H_t)\) cancels those treatment-probability factors. The remaining factors are the covariate and outcome distributions under the intervention that sets each \(A_t=a_t\). Consistency and sequential exchangeability identify those factors with the corresponding distributions of \(L_{t+1}^{\bar a}\) and \(Y^{\bar a}\); summing over the covariate histories therefore gives \(E[Y^{\bar a}]\). Positivity ensures the divisions are defined. Thus the probabilities that cancel are precisely the probabilities conditional on the pre-decision histories \(H_t\), not probabilities conditional on future measurements. \(\square\)

Worked Examples: Classifying Measurements by Decision Time

Worked Example: How Future-Covariate Adjustment Distorts Weights

Practical Checks and Common Failure Modes

Before fitting a treatment model, construct a decision-specific timeline. Record when each treatment was assigned, when each candidate covariate was measured, and which measurements were available at each decision. A visit label alone may not resolve the order. If the order is ambiguous, check the protocol, source records, or data-generation process rather than assuming the covariate preceded treatment.

A common mistake is to include every measurement that improves prediction of treatment. The reasoning is that a more predictive model must give better weights. The correction is that prediction quality does not establish that the model conditions on the right information. A future measurement can be highly predictive precisely because it was affected by treatment. The relevant model is the one for assignment conditional on the pre-decision history, subject to the causal assumptions needed for weighting.

Cross-fitting, flexible machine-learning algorithms, calibration, and smoothing do not fix temporal leakage: they can estimate an incorrectly defined conditional probability very accurately. Conversely, a value recorded at the same visit may be appropriate if it was definitely measured before assignment. Timing, not the variable name or visit number, is decisive. When actual ordering is unavailable, conclusions may depend on unverifiable assumptions about that ordering.

This timing rule complements, rather than replaces, the standard assumptions behind inverse probability weighting. It does not guarantee no unmeasured confounding, positivity, correct measurement, or a well-defined treatment strategy. It identifies which observed history is eligible to condition the treatment model; the causal assumptions still require separate justification.