Why an Empty Treatment Arm Matters
To learn a causal effect from observational data, it is not enough to adjust for measured covariates and assume that treatment assignment is unconfounded within covariate strata. We also need treatment variation in the strata that contribute to the target population. If everyone in a stratum receives treatment, the data contain no untreated outcomes for that stratum. The missing outcomes are not merely difficult to estimate: without additional assumptions or information, the observed data do not determine them.
This is an identification problem. A causal quantity is identified if the observed-data distribution, together with the stated assumptions, determines one value for that quantity. To show that a quantity is not identified, it is enough to construct two possible populations that satisfy the assumptions and have exactly the same observed-data distribution, but different values of the causal quantity.
Consistency links the observed outcome to the potential outcome under the treatment actually received: \(Y=Y(A)\). Conditional exchangeability, \(Y(a)\perp A\mid X\), says that within levels of \(X\), treatment assignment carries no further information about the potential outcome under level \(a\). These assumptions play different roles. Exchangeability addresses comparability of people assigned to different treatment levels; positivity ensures that both levels actually occur for comparison. When an arm is absent, observed outcomes cannot reveal the missing arm’s conditional mean, even if exchangeability is assumed.
A Construction Showing Nonidentification
Consider two covariate strata, \(X=0\) and \(X=1\), each making up half the target population. Suppose both treatment levels occur when \(X=0\), but everyone with \(X=1\) receives treatment. The conditional average effect in the first stratum can be learned under consistency and conditional exchangeability. In the second stratum, the data reveal outcomes under treatment but say nothing directly about outcomes under no treatment.
The following construction makes the consequence explicit. In both possible populations, the \(X=0\) stratum has \(E[Y(1)\mid X=0]=5\) and \(E[Y(0)\mid X=0]=2\). In the \(X=1\) stratum, everyone has \(A=1\) and \(Y(1)=8\). The two populations differ only in \(Y(0)\) for that stratum.
Worked Example: Two Compatible Populations, Two Different Effects
In Population I, set \(Y(0)=0\) for everyone with \(X=1\). In Population II, set \(Y(0)=6\) for everyone with \(X=1\). Treatment assignment and all other potential outcomes are the same in both populations.
In both populations, the observed outcomes in the \(X=0\) stratum have means 5 for treated people and 2 for untreated people. In the \(X=1\) stratum, everyone is treated and has outcome 8. Thus the observed-data distributions are identical: the value of \(Y(0)\) for \(X=1\) is never observed.
The average effect in the \(X=0\) stratum is \(5-2=3\). In Population I, the effect in the \(X=1\) stratum is \(8-0=8\), so the population average effect is
In Population II, the \(X=1\) effect is \(8-6=2\), giving
Both populations obey consistency and can obey conditional exchangeability. In the \(X=1\) stratum, treatment is constant, so conditional exchangeability does not create an observed untreated comparison; it does not restrict the unobserved \(Y(0)\) values enough to distinguish these populations. Since the observed data are the same but the average effects differ, the population average effect is not identified under these assumptions.
The calculation illustrates the general logic. If the target population gives positive weight to a stratum where one treatment level never occurs, the observed data leave the corresponding potential-outcome distribution unconstrained. Changing that distribution can change the stratum’s effect and, therefore, the target population’s average effect. This is a failure of identification, not just a large standard error or a small-sample inconvenience.
Which Missing Outcome Is Unidentified?
The problem is symmetric. A stratum with no untreated people leaves \(E[Y(0)\mid X=x]\) unobserved; a stratum with no treated people leaves \(E[Y(1)\mid X=x]\) unobserved. The resulting uncertainty can be substantial even when the outcome is binary and therefore bounded. If outcomes are unbounded, the data and these assumptions alone may not give a finite range for the missing mean.
Worked Example: A Binary Outcome and a Range of Possible Effects
Suppose \(X=0\) and \(X=1\) each account for half the target population. In the \(X=0\) stratum, everyone is untreated, and the observed outcome mean is \(0.2\). In the \(X=1\) stratum, both treatment levels occur; assume conditional exchangeability, and suppose the observed conditional means are \(E[Y(1)\mid X=1]=0.7\) and \(E[Y(0)\mid X=1]=0.4\). The outcome is binary.
The \(X=1\) effect is \(0.7-0.4=0.3\). In the \(X=0\) stratum, the observed data establish \(E[Y(0)\mid X=0]=0.2\), but there are no treated people. Because a binary outcome mean must lie between 0 and 1, the unobserved mean \(E[Y(1)\mid X=0]\) can range from 0 to 1 without changing the observed outcomes.
If the missing mean is 0, the \(X=0\) effect is \(0-0.2=-0.2\), and the population effect is
If instead the missing mean is 1, the \(X=0\) effect is \(1-0.2=0.8\), and the population effect is
The observed-data summaries are identical under these alternatives. The assumptions and the binary outcome range constrain the average effect to the interval from 0.05 to 0.55 in this example, but they do not select one value. The effect is therefore not point identified.
Structural Violations Versus Empty Sample Cells
A zero observed count in a treatment-by-covariate cell is a warning, but it is not by itself proof that the population probability of treatment in that stratum is zero. A population may have positive treatment probability while a finite sample happens to contain no people assigned to that treatment. The distinction matters: a structural zero prevents identification from the population observed-data distribution under the stated assumptions, whereas a sample zero can be a chance feature of a particular data set.
Worked Example: An Empty Cell Despite Population Positivity
Suppose a stratum contains \(n=8\) sampled people and, in the population, each has treatment probability \(P(A=1\mid X=x)=0.2\). Assume independent treatment assignments for this calculation. Population positivity holds because the treatment probability is between 0 and 1. The probability that no sampled person in this stratum is treated is
Thus there is about a 16.78% chance of an empty treated cell in this sample, despite positive population probability of treatment. The observed cell cannot supply a treated outcome mean for this sample, so a direct within-cell estimate is unavailable. But this sample result alone does not establish a structural positivity violation in the population. Conversely, a model-based estimate of the treatment probability should not be treated as proof that the missing cell contains usable outcome information.
Diagnostics should therefore examine both the design and the data. Tabulate treatment counts across relevant covariate strata, inspect estimated treatment probabilities, and ask whether empty cells reflect the assignment mechanism or limited sampling. Estimates near zero or one also deserve attention: even without a literal zero, comparisons rely on very few observations from one arm and can be highly sensitive to modeling choices. Such near-violations raise practical instability; they are not identical to a population probability of exactly zero.
What Can Be Done, and What Cannot
A modeling procedure can produce a number for an unsupported stratum by extrapolating from other covariate values or by imposing a functional form. That number is not determined by the observed outcomes in the missing arm. Its interpretation depends on the added model restrictions, which should be stated and justified rather than mistaken for information supplied by the data.
Other responses change the question or the information available. Researchers can restrict the target population to covariate strata with treatment overlap, collect data that include both treatment levels in the unsupported strata, or conduct sensitivity analyses under explicit assumptions about the missing potential outcomes. Restricting the target population may leave an effect identifiable for that overlap population, but it does not identify the effect for the original target population.
Worked Example: An Effect for an Overlap Population
Return to two equally prevalent strata. In \(X=0\), suppose both treatment levels occur, with conditional outcome means 5 under treatment and 2 without treatment. In \(X=1\), everyone is treated, with observed mean outcome 8. If the target is the population with \(X=0\), then consistency and conditional exchangeability, along with treatment overlap in that stratum, identify its mean effect as
That conclusion is about the \(X=0\) target population only. It cannot be extended to the full population without information or assumptions about the missing \(Y(0)\) outcomes in \(X=1\). The two-population construction above shows why: those assumptions can change the full-population effect while leaving the overlap-stratum effect and all observed data unchanged.
This is the central limitation: positivity is about whether the observed data contain the treatment comparisons required for the particular target estimand. A well-fitting model, a large overall sample, or adjustment for many covariates cannot recover a comparison that the assignment mechanism never permits. An analysis should report which strata lack support and be clear if it changes the target population or relies on extrapolation.
Common Mistakes
- “Exchangeability is enough.” Exchangeability addresses confounding within strata; it does not supply outcomes under a treatment level that is absent from a stratum.
- “The missing outcome can be filled in by the observed outcome.” Consistency equates the observed outcome to the potential outcome under the treatment received, not to the unobserved potential outcome under the other treatment.
- “An empty sample cell proves a structural violation.” A finite sample can have an empty cell even when both population treatment probabilities are positive.
- “A model’s extrapolated prediction is identified by the data.” Predictions outside observed treatment support rely on added modeling assumptions; they do not remove the identification problem under consistency and exchangeability alone.
- “An effect estimated in overlap strata is the full-population effect.” It applies to the restricted target population unless further assumptions justify extending it.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a stratum where everyone receives treatment, which potential-outcome mean is unobserved?
- Two populations have identical observed data and satisfy the same stated assumptions, but their average treatment effects differ. What does this establish?
- For a binary outcome, a stratum has observed \(E[Y(0)\mid X=x]=0.3\), but no treated people. What range can the stratum’s average effect take based on the outcome range alone?
- Does a sample with zero treated people in a stratum necessarily imply that population positivity fails? Explain.
- Why does estimating an effect in an overlap subpopulation not automatically identify the effect in the full target population?
Solutions. (1) \(E[Y(0)\mid X=x]\), because no one in that stratum reveals an outcome under no treatment. (2) The average treatment effect is not identified by the observed data and assumptions: identification requires one value across all compatible populations. (3) Since \(E[Y(1)\mid X=x]\) can range from 0 to 1, the effect \(E[Y(1)\mid X=x]-0.3\) can range from \(-0.3\) to \(0.7\). (4) No. A finite sample may have an empty cell by chance even when the population probability of treatment is positive. (5) The overlap estimate concerns a different, restricted target; it does not reveal the missing potential-outcome mean in unsupported strata included in the full population.