Why the Same Data Can Support Different Causal Effects
An observed association between a later treatment and an outcome does not by itself reveal how much of that association is causal. A common difficulty is sequential confounding: a factor related to both later treatment assignment and the outcome is not adequately controlled in the treatment history. The resulting problem is not merely that an estimator is biased. Without a valid identifying assumption, different causal explanations can agree on every feature of the observed data while disagreeing about the effect of a treatment regime.
The construction below uses two treatment decisions. The first treatment \(A_0\) is randomized and has no effect in either model. The later treatment is \(A_1\), and the binary outcome is \(Y\). There is no measured intermediate covariate; equivalently, any measured intermediate history \(L\) is constant. The regime \((0,1)\) sets \(A_0=0\) and \(A_1=1\), while \((0,0)\) sets both treatments to zero.
Here \(Y^{a_0,a_1}\) denotes the outcome that would be observed if treatment were set to \(a_0\) at the first decision and \(a_1\) at the second. Consistency means that, for a person who actually follows \((A_0,A_1)=(a_0,a_1)\), the observed outcome equals \(Y^{a_0,a_1}\). Sequential exchangeability requires, among its conditions, that treatment at each decision be conditionally independent of the relevant future potential outcomes given the observed treatment and covariate history. Positivity requires positive probability of each treatment choice being considered at the histories to which the regime is applied.
When the assumptions for the two-time identification formula hold, it expresses a regime mean using observed conditional outcome distributions and the covariate distribution under the regime. Its crucial step is replacing a conditional mean of a potential outcome by an observed conditional mean among people who received that treatment. Sequential exchangeability justifies that replacement. If it fails, the observed conditional mean need not equal the counterfactual mean, even when consistency and positivity hold.
Two Models With the Same Observed Distribution
We now specify the models completely. All variables are binary except the independent uniform random variables used to generate them. In the confounded model, an unobserved binary variable \(U\) affects both later treatment and potential outcomes. The construction will retain positive probability of both later treatment choices at either value of \(U\).
Worked Example: A Randomized Model With No Regime Effect
In Model R, let \(A_0,U,A_1,E_0,E_1\) be mutually independent, with \(A_0,U,A_1\) each Bernoulli with probability \(1/2\), and \(E_0,E_1\) each Uniform\((0,1)\). Define the potential outcomes by
Thus \(A_1\) is independent not only of \(A_0\) and \(U\), but also of both random variables generating the potential outcomes. The observed outcome is \(Y=Y^{A_0,A_1}\). For either value of \(A_1\), its outcome probability is \(P(Y=1\mid A_1)=1/2\), and \(P(A_1=1)=1/2\). The same is true conditional on \(A_0\), since \(A_0\) is independent of the other variables.
Under either regime \((0,1)\) or \((0,0)\), the outcome has mean \(1/2\). Therefore
The treatment choices have positive probability at both decision times. In particular, \(P(A_0=0)=1/2\) and \(P(A_1=a_1\mid A_0=0)=1/2\) for \(a_1=0,1\). The model satisfies consistency by defining the observed outcome as the potential outcome under the treatment actually received.
Worked Example: A Confounded Model With a Different Regime Effect
In Model C, let \(A_0,U,E_0,E_1,R\) be mutually independent, with \(A_0\) and \(U\) Bernoulli with probability \(1/2\), and \(E_0,E_1,R\) Uniform\((0,1)\). Define
so \(P(A_1=1\mid U=0)=1/4\) and \(P(A_1=1\mid U=1)=3/4\). Define potential outcomes using the following probabilities:
| Value of \(U\) | \(P(Y^{a_0,1}=1\mid U)\) | \(P(Y^{a_0,0}=1\mid U)\) |
|---|---|---|
| 0 | 0.2 | 0.4 |
| 1 | 0.6 | 0.8 |
Formally, set \(q_{1,0}=0.2,\ q_{1,1}=0.6,\ q_{0,0}=0.4,\ q_{0,1}=0.8\), and let \(Y^{a_0,a_1}=1\{E_{a_1}\leq q_{a_1,U}\}\). The potential outcomes do not depend on \(a_0\). The observed outcome is again \(Y=Y^{A_0,A_1}\).
First check positivity and the observed treatment distribution. Since \(P(U=1)=P(U=0)=1/2\),
Bayes' rule gives \(P(U=1\mid A_1=1)=(\frac12\cdot\frac34)/(\frac12)=3/4\), and \(P(U=1\mid A_1=0)=(\frac12\cdot\frac14)/(\frac12)=1/4\). Both treatment probabilities conditional on either value of \(U\) are positive. As \(A_0\) is an independent fair coin, both values of \(A_0\) also have positive probability.
Now calculate the observed outcome probability for each later treatment. Among people with \(A_1=1\), the probability of \(U=1\) is \(3/4\), so
Among people with \(A_1=0\), the probability of \(U=1\) is \(1/4\), giving
Because \(A_0\) is independent and does not affect treatment assignment or the outcome, these equalities also hold conditional on \(A_0\). Thus the observed outcome is a fair binary variable at either value of \(A_1\).
The regime means, however, average over the population distribution of \(U\), not the different distributions of \(U\) among the observed treatment groups. They are
Consequently the regime contrast is \(0.4-0.6=-0.2\), not zero. Model C also satisfies consistency by construction. The discrepancy from Model R is not caused by inconsistent treatment versions or by absent treatment support: it arises because \(U\), which predicts both \(A_1\) and potential outcomes, is unobserved.
Why Sequential Exchangeability Fails
In Model C, \(A_1\) depends on \(U\), and the potential outcomes depend on \(U\). The dependence can be verified numerically rather than inferred from the data-generating description. For example, the marginal mean of \(Y^{0,1}\) is \(0.4\), while among those observed to receive \(A_1=1\), its conditional mean is
Since \(0.5\ne 0.4\), \(Y^{0,1}\) is not independent of \(A_1\). Likewise, \(E[Y^{0,0}\mid A_1=0]=0.5\), whereas \(E[Y^{0,0}]=0.6\). With \(L\) constant, conditioning on the observed history does not remove this dependence. Sequential exchangeability at the later decision therefore fails.
Proof. An identified contrast must be a unique function of the observed-data distribution under the stated assumptions. Models R and C induce the same distribution, so any such function must take the same value in both. But direct calculation gives contrast \(0\) in Model R and \(-0.2\) in Model C. No single value can equal both. Therefore the observed distribution and stated restrictions do not identify the contrast. \(\square\)
The common observed distribution can be stated exactly: in both models, \(A_0,A_1,Y\) are mutually independent Bernoulli\((1/2)\) variables. For Model C, the calculations above establish that \(A_1\) is fair and \(P(Y=1\mid A_1)=1/2\); independence of \(A_0\) gives the full factorization. Model R gives the same factorization directly from its construction.
Worked Example: What the Observed Regime Calculation Returns
Suppose an analyst uses the observed conditional outcome probabilities to compare \((0,1)\) with \((0,0)\). In both models, \(P(Y=1\mid A_0=0,A_1=1)=1/2\) and \(P(Y=1\mid A_0=0,A_1=0)=1/2\). Thus the observed-data comparison is
This agrees with the true regime contrast in Model R, but not with the contrast \(-0.2\) in Model C. In Model C, the observed groups differ in their \(U\) composition: \(P(U=1\mid A_1=1)=3/4\), while \(P(U=1\mid A_1=0)=1/4\). The conditional outcome probabilities combine the treatment-specific potential outcome probabilities with these different compositions. They are not the population-average counterfactual means, which use \(P(U=1)=1/2\) under each intervention.
This derivation identifies the exact unsupported substitution: the observed conditional mean among those with \(A_1=a_1\) is used in place of the counterfactual mean under \(A_1=a_1\). The observed equality of the conditional outcome probabilities does not validate that substitution.
What the Example Does and Does Not Establish
The example shows that the observed distribution alone cannot distinguish a no-effect randomized explanation from a confounded explanation with a nonzero regime effect. It does not show that every violation of sequential exchangeability makes every causal estimand unidentified. Some estimands can remain identified under additional structure, and a particular violation might not alter a particular contrast. The conclusion here is specific and rigorous: without an assumption or additional information that rules out one of these compatible models, this contrast is not uniquely determined.
An analyst cannot generally diagnose sequential exchangeability by checking whether the observed treatment and outcome are associated. Here they are independent in the observed data even though the confounded model has a nonzero causal effect. Conversely, an observed association can arise under a causal effect, confounding, or both. Assessing exchangeability therefore depends on study design, temporal knowledge, measured confounders, and substantive reasoning about what influences treatment and outcome.
The main alternative is to justify exchangeability using a sufficiently rich observed history and then apply the identification formula for that history. Randomized treatment at each decision provides a different reference design because treatment assignment is independent of the relevant potential outcomes by design. If unmeasured confounding remains plausible, the observed data do not supply the missing restriction; conclusions require additional assumptions or information. Neither a more flexible outcome model nor a more elaborate weighting calculation can, by itself, recover a unique causal effect from observationally indistinguishable models.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In Model C, calculate \(P(A_1=0\mid U=1)\) and \(P(A_1=1\mid U=0)\). Does positivity fail?
- Calculate \(E[Y^{0,1}]\) and \(E[Y^{0,0}]\) in Model C, and state their contrast in the order \((0,1)\) minus \((0,0)\).
- Why do Models R and C have the same observed distribution of \(A_0,A_1,Y\), despite their different regime contrasts?
- Would observing that \(A_1\) and \(Y\) are independent establish sequential exchangeability? Explain using the construction.
Solutions. (1) \(P(A_1=0\mid U=1)=1-3/4=1/4\), and \(P(A_1=1\mid U=0)=1/4\). Both are positive, as are the complementary treatment probabilities, so positivity does not fail. (2) The population distribution has \(P(U=1)=P(U=0)=1/2\). Hence \(E[Y^{0,1}]=\frac12(0.6)+\frac12(0.2)=0.4\), and \(E[Y^{0,0}]=\frac12(0.8)+\frac12(0.4)=0.6\). The contrast is \(0.4-0.6=-0.2\). (3) Both models give fair marginal distributions for \(A_0,A_1,Y\), with all three mutually independent. Model C obtains the fair observed outcome rates by mixing different \(U\)-specific potential outcome rates with different \(U\) compositions across treatment groups. Model R obtains the same rates under randomized later treatment. (4) No. Model C has \(A_1\) and \(Y\) independent in the observed distribution, but \(A_1\) is associated with the potential outcomes through \(U\). Observed independence therefore does not establish sequential exchangeability.