Unmeasured Confounding at a Later Decision

Why a Good First Decision Is Not Enough

In a longitudinal study, treatment is assigned more than once. Randomization, or a credible exchangeability assumption, at the first decision protects comparisons made at that time. But it does not automatically protect the comparison at a later decision. A variable that predicts both the later treatment and the outcome can make people receiving different later treatments incomparable, even if the first treatment was assigned independently of their potential outcomes.

The important distinction is between an assumption at each decision and an assumption about the study as a whole. Let \(A_1\) and \(A_2\) be binary treatments at two decision times, and let \(Y(a_1,a_2)\) denote the outcome that would be observed under treatment history \((a_1,a_2)\). First-decision exchangeability concerns \(A_1\). Later-decision exchangeability concerns \(A_2\), conditional on the observed history available before that decision. The first condition does not imply the second.

This tutorial focuses on that specific failure. The counterexample will satisfy consistency, have treatment support at both decisions, and make \(A_1\) exchangeable. Nevertheless, its observed data will not determine the causal effect of \(A_2\). Consistency means that the observed outcome equals the potential outcome under the treatment history actually received: \(Y=Y(A_1,A_2)\). Treatment support means that each treatment value occurs with positive probability in the histories considered.

A Two-Period Counterexample

Consider a population with no measured covariates. Let \(A_1\) be a fair random treatment, \(P(A_1=1)=1/2\). The later outcome is binary. We compare two causal models, each of which generates the same observed distribution of \((A_1,A_2,Y)\). In the first model, an unmeasured variable \(U\), present just before the second decision, predicts both \(A_2\) and \(Y\). In the second, the later treatment is randomized and has a causal effect.

The observed association is substantial, although the causal effect in Model I is zero. The reason is that the observed treatment groups differ in \(U\). Indeed, \(U=1\) is more common among those who receive \(A_2=1\), and \(U=1\) also guarantees \(Y=1\).

First Exchangeability Can Still Hold

In Model I, all potential outcomes are functions of \(U\), and \(A_1\) was generated independently of \(U\). Therefore \(A_1\) is independent of the entire collection \(\{Y(a_1,a_2):a_1,a_2\in\{0,1\}\}\). This verifies first-decision exchangeability directly. At the same time, \(A_2\) depends on \(U\), and \(Y(a_1,a_2)=U\), so \(A_2\) is associated with the potential outcome even after conditioning on \(A_1\).

Why the Causal Effect Is Not Identified

A causal quantity is identified from the observed distribution if every causal model satisfying the stated assumptions and producing that observed distribution gives the quantity the same value. Showing two compatible models with different values is enough to disprove identification. Here the first model attributes the observed association to \(U\); the second can attribute it to a causal effect of \(A_2\).

To prove the claim, retain the same fair \(A_1\) in Model II. Generate \(A_2\sim\operatorname{Bernoulli}(1/2)\), independently of \(A_1\) and a random variable \(V\sim\operatorname{Uniform}(0,1)\). Define

Here \(A_1\) is independent of all potential outcomes, as in Model I. Under consistency, \(Y=Y(A_1,A_2)\). For \(A_2=1\), the outcome risk is \(P(V\leq0.8)=0.8\); for \(A_2=0\), it is \(P(V\leq0.2)=0.2\). Since \(A_2\) is a fair coin independent of \(V\), the four joint probabilities of \((A_2,Y)\) are \(0.4, 0.1, 0.1, 0.4\), in the same cells as Model I. Independence and the same distribution of \(A_1\) then make the full observed distribution of \((A_1,A_2,Y)\) identical in the two models.

But in Model II, for either fixed \(a_1\),

whereas the same contrast is \(0.5-0.5=0\) in Model I. Both models satisfy consistency, first-decision exchangeability, and the stated treatment support. They agree on every observed-data probability but disagree on the causal contrast. Therefore that contrast is not identified under those assumptions. The differing values are possible because first-decision exchangeability does not impose later-decision exchangeability. \(\square\)

What Can Be Checked, and What Cannot

The example separates several issues that are easy to conflate. Consistency says that observed outcomes agree with the potential outcomes under the treatments actually received. Positivity says that the relevant treatment options occur in the observed histories. Exchangeability says that, conditional on the measured history, treatment assignment is unrelated to the potential outcomes. The first two properties do not entail the third.

Observed data can reveal some warning signs: measured histories may have very different treatment rates, outcome patterns may be sensitive to adjustment choices, or treatment may be nearly absent in some measured strata. Those findings can point to practical problems, but no diagnostic based only on the observed distribution can establish that an unmeasured common cause is absent. In the counterexample, the two models have exactly the same observed distribution, so every diagnostic computed from that distribution has the same result under both.

The main comparison is between a claim about the first decision and a claim about every decision. Randomizing \(A_1\), or adjusting for baseline variables that make \(A_1\) exchangeable, does not control a variable that predicts \(A_2\) and \(Y\) later. Adjusting for measured history can address confounding represented in that history, but it cannot directly adjust for \(U\) when \(U\) is unavailable. A design that randomizes the later treatment would remove this particular assignment pathway; otherwise, substantive knowledge, richer measurement, and sensitivity analysis are needed to assess how conclusions depend on the unmeasured factor.

The counterexample establishes a limitation, not a method for repairing it: observed data cannot, without additional assumptions, determine which of the two causal explanations is correct. It also does not say that later exchangeability always fails. It shows why that condition must be defended separately rather than inferred from first-treatment exchangeability, consistency, or support.