Identified Sets And Partial Identification

Why a Causal Effect May Be a Set

An observational study can estimate an association between treatment and outcome quite precisely while leaving the causal effect uncertain. The reason is that the observed data do not generally reveal what would have happened to each person under the treatment they did not receive. When assumptions do not determine one unique causal effect, the appropriate target is not a single identified value but a set of values compatible with the data and assumptions.

An identified set is a population-level object: it describes what the observed-data distribution and a specified model permit, not how much sampling error a particular estimate has. Its endpoints therefore depend on both the information in the observed distribution and the restrictions imposed on the unobserved parts of the causal model. A narrower set can reflect stronger assumptions, not necessarily more informative data.

Definition: Let $P_O$ denote the population distribution of the observed variables, let $\mathcal{M}(P_O,A)$ be the set of causal models that reproduce $P_O$ and satisfy assumptions $A$, and let $\psi(M)$ be the causal effect in model $M$. The identified set is $$ \Psi_I(P_O;A)=\{\psi(M):M\in\mathcal{M}(P_O,A)\}. $$ It contains every causal effect value compatible with the observed distribution and assumptions. A parameter is point identified when this set contains exactly one value; it is partially identified when more than one value remains possible.

The Causal Quantity and the Compatible Models

For clarity, consider a binary treatment $A$, with $A=1$ denoting treatment and $A=0$ denoting no treatment. Let $Y(1)$ and $Y(0)$ be the potential outcomes for a person under treatment and no treatment, respectively. A common target is the average treatment effect $ \psi=E[Y(1)-Y(0)] $ in a specified target population. The observed outcome is $Y=Y(A)$, which is the consistency condition: the outcome recorded under the treatment actually received equals the corresponding potential outcome.

The observed distribution contains $A$, $Y$, and any measured covariates. It does not ordinarily contain both potential outcomes for the same person, nor does it directly reveal all common causes of treatment and outcome. A causal model specifies a joint distribution for the observed and unobserved quantities. Compatibility requires that, after the unobserved quantities are marginalized out, the model yields the observed distribution $P_O$. The assumptions further restrict which such models are allowed.

Thus an identified set is obtained by asking a model-based question: among all causal models that fit what is observed and obey the declared assumptions, which values of $\psi$ occur? If the answer is one value, the effect is point identified. If it is several values or a range, the data and assumptions have not singled out one effect. If there is no compatible model, the assumptions and observed distribution contradict one another.

Endpoints and What They Mean

When the identified set is bounded, its lower and upper endpoints can be written $ \underline{\psi}=\inf\Psi_I $ and $ \overline{\psi}=\sup\Psi_I $. If the set is an interval, these endpoints give its full range. But an identified set need not be an interval: it may have gaps, consist of separate values, or be unbounded. The interval from the infimum to the supremum is then an outer range, not necessarily the exact set.

An endpoint is meaningful only relative to the assumptions used to define compatibility. For example, one assumption set may allow a confounding contribution anywhere in a broad range, while a stronger set may restrict that contribution more tightly. The latter can produce narrower endpoints. That change is a consequence of the additional restriction; it is not evidence that the observed data alone have become more informative.

Worked Example: A Bound on the Confounding Contribution

Suppose hypothetical population data show a treated-group mean outcome of 18 units and an untreated-group mean of 14 units. The observed mean difference is $ \delta=18-14=4 $ units. For illustration, suppose the causal model expresses this difference as $ \delta=\psi+\gamma $, where $\psi$ is the average causal effect and $\gamma$ is the contribution of confounding to the observed difference. Assume the only restriction on $\gamma$ is $ -1\leq\gamma\leq2 $ units.

For every allowed model, solve the decomposition for the causal effect:

\psi=\delta-\gamma=4-\gamma.

Because $\gamma$ can be as large as 2, the smallest compatible effect is $4-2=2$ units. Because $\gamma$ can be as small as $-1$, the largest compatible effect is $4-(-1)=5$ units. Every intermediate value is allowed under the stated model: for any $\psi$ between 2 and 5, setting $\gamma=4-\psi$ gives a value between $-1$ and 2.

Therefore the identified set under these assumptions is $[2,5]$ units. The calculation does not establish that the true effect is 3.5 units, or that it is equally likely to be any value in the range. It establishes that the observed difference and the stated confounding restriction do not distinguish among those effect values.

A Short Derivation: Mapping Restrictions Into Effect Values

The previous calculation is an instance of a general mapping. Suppose the observed distribution determines a quantity $d(P_O)$, and the causal model implies $d(P_O)=\psi+g$, where $g$ is an unobserved nuisance quantity. If assumptions restrict $g$ to a set $G$, then the compatible effect set is the image of $G$ under the transformation $g\mapsto d(P_O)-g$:

\Psi_I(P_O;A)=\{d(P_O)-g:g\in G\}.

If $G=[g_L,g_U]$, subtraction reverses endpoint order. For any $g\in[g_L,g_U]$, we have $d(P_O)-g_U\leq d(P_O)-g\leq d(P_O)-g_L$. Both outer values are attained when $g=g_U$ and $g=g_L$, respectively. Hence

\Psi_I(P_O;A)=[d(P_O)-g_U,\ d(P_O)-g_L].

This derivation relies on the stated decomposition and on allowing every $g$ in the interval. If the assumptions permit only selected values of $g$, the resulting effect set may have gaps. If $G$ has no finite endpoints, the effect set may also be unbounded. The algebra does not justify the restriction $G$; that restriction must be defended as a substantive assumption.

Worked Example: The Same Data, Different Assumptions

Keep the hypothetical observed difference $d(P_O)=4$ units and the decomposition $4=\psi+\gamma$. Compare three alternative assumption sets. Each one is an assumption about the confounding contribution, not a change to the observed data.

Broad restriction. If $-1\leq\gamma\leq2$, then $ \psi=4-\gamma\in[4-2,4-(-1)]=[2,5] $ units.

Narrower restriction. If substantive knowledge instead supports $0\leq\gamma\leq1$, then $ \psi\in[4-1,4-0]=[3,4] $ units. This set is narrower because the assumption rules out more confounding contributions.

Point restriction. If the model assumes $\gamma=0$, then $ \psi=4-0=4 $ units. This is point identification under that assumption. It is not a conclusion obtained from the observed difference alone.

The comparison shows why a reported identified set must be accompanied by its assumptions. From the same observed contrast, the three assumptions yield three different answers. Choosing among them requires scientific justification; arithmetic cannot determine which assumption is credible.

Identified Sets, Estimates, and Sampling Uncertainty

The population set $\Psi_I(P_O;A)$ is defined using the population observed-data distribution. In practice, that distribution is unknown and is estimated from a sample. A sample calculation may estimate the set’s endpoints, but those estimates vary from sample to sample. This creates sampling uncertainty in addition to the non-identification represented by the set itself.

For example, suppose a sample estimates an observed difference as $\hat{\delta}=4.0$ units with standard error $SE=0.5$ units. Under a suitable large-sample normal approximation, a 95% confidence interval for the population difference is approximately $4.0\pm1.96(0.5)=[3.02,4.98]$ units. If the assumption remains $-1\leq\gamma\leq2$, transforming the two endpoints gives the conservative range $[3.02-2,\ 4.98-(-1)]=[1.02,5.98]$ units for effects compatible with values in that confidence interval and the stated restriction.

This transformed range is not the population identified set itself: it incorporates uncertainty about the observed difference. Nor does it mean there is a 95% probability that the fixed causal effect lies in the interval. Its coverage statement concerns repeated-sampling performance under the approximation and the assumed model. A rigorous confidence procedure for partially identified parameters may require methods tailored to the endpoint estimator; the simple transformation is illustrative, not universally valid.

Worked Example: A Discrete Identified Set Need Not Fill Its Endpoints

Suppose a hypothetical observed contrast is again 4 units and the causal model implies $4=\psi+\gamma$. This time assume the model permits only two confounding contributions, $\gamma\in\{1,3\}$, rather than every value between them.

If $\gamma=1$, then $\psi=4-1=3$ units. If $\gamma=3$, then $\psi=4-3=1$ unit. The exact identified set is therefore $\{1,3\}$ units, not the interval $[1,3]$. Its infimum and supremum are 1 and 3 units, but an effect of 2 units is not compatible with this particular model.

Reporting only $[1,3]$ would still give an outer range containing all compatible effects, but it would add values that the assumptions exclude. This distinction matters whenever the permitted model space is discrete or otherwise non-convex.

How to Assess a Reported Set

A clear analysis makes both the observed-data information and the assumptions visible. To assess a reported identified set, ask:

What causal quantity is targeted, and for which population?
What observed-data distribution or summary constrains the causal models?
Which assumptions restrict unobserved outcomes, confounding, or other latent features?
Are the reported endpoints attainable, or are they only an outer range?
Does the result describe a population identified set, a sample estimate of that set, or a confidence region accounting for sampling error?

Diagnostics can reveal problems but do not by themselves identify a causal effect. Negative controls and falsification analyses may challenge aspects of a causal model or expose patterns inconsistent with it. Passing such checks does not prove that all identifying assumptions hold, and it does not automatically supply numerical endpoint restrictions. The restrictions used to form an identified set still need explicit justification.

Compared with a point-estimation approach, partial identification avoids pretending the data and assumptions select one causal value when they do not. Compared with a sensitivity analysis that reports effects across a range of chosen sensitivity parameters, an identified-set analysis emphasizes the full collection of effect values allowed by a specified model. These approaches can overlap: a sensitivity restriction can define the set. Neither approach removes the need to explain why its assumptions are relevant.

Common Mistakes and Limitations

Calling an identified set an estimate. The set is a population feature of the model and observed distribution; a sample provides estimates of its features.
Treating endpoints as equally plausible values. A set records compatibility, not a probability distribution over effects.
Reporting a range without its assumptions. Endpoints have no standalone interpretation; they are conditional on the restrictions that generated them.
Assuming the set must be an interval. A set can be disconnected. An interval between its extrema may include values that are not compatible.
Confusing identification uncertainty with sampling uncertainty. More data can reduce sampling uncertainty without narrowing the population identified set if the same assumptions leave multiple models observationally equivalent.

Partial identification is limited by the credibility and informativeness of the assumptions. Weak assumptions may leave a broad or unbounded set. Strong restrictions may yield a precise-looking narrow set while excluding plausible causal models. A useful report therefore presents the assumptions, the set, and a sensitivity discussion of how the result changes under alternative defensible restrictions.

Check Your Understanding

Answer each question using the definitions and calculations in this tutorial.

In one sentence, distinguish an identified set from a point estimate.
In the decomposition $d=\psi+\gamma$, let $d=6$ units and assume $1\leq\gamma\leq2.5$. Find the identified set for $\psi$.
If the only allowed values of $\gamma$ in the previous question are 1 and 2.5, what is the exact identified set? Is it the full interval between its extrema?
Can a narrow confidence interval for an observed association guarantee a narrow identified set for the causal effect? Explain.

Solutions. (1) A point estimate is one computed value from a sample, whereas an identified set is the collection of population causal values compatible with the observed distribution and assumptions. (2) Since $\psi=6-\gamma$, the minimum is $6-2.5=3.5$ and the maximum is $6-1=5$, so the set is $[3.5,5]$ units. (3) The exact set is $\{6-1,6-2.5\}=\{5,3.5\}$ units, not the full interval, because intermediate values of $\gamma$ are excluded. (4) No. A narrow confidence interval can reflect precise estimation of the observed association, while weak assumptions may still permit a broad range of causal effects.