Instrument Independence in Observational Settings

Why Independence Needs Special Attention in Observational IV

In a randomized encouragement design, the assignment mechanism can provide a direct reason to regard the instrument as independent of participants’ underlying treatment responses and outcomes. In an observational setting, the instrument is not assigned by a known random mechanism. The same characteristics that influence which instrument value a person receives may also predict how that person would respond to treatment. An association between the instrument and the outcome can then reflect differences in who received each instrument value, rather than the variation in treatment receipt that IV analysis needs.

For this tutorial, let $Z$ be a binary instrument, $D$ the treatment received, and $Y$ the outcome. Write $D(z)$ for the treatment a unit would receive if the instrument were set to $z$, and $Y(d)$ for the outcome that unit would have under treatment value $d$. The independence assumption concerns the whole collection of potential outcomes and potential treatment receipts, not merely the observed outcome or treatment.

Definition: Unconditional instrument independence requires $$ Z \perp \{Y(0),Y(1),D(0),D(1)\}. $$ If $X$ is a set of covariates measured before the instrument is assigned or received, conditional instrument independence requires $$ Z \perp \{Y(0),Y(1),D(0),D(1)\}\mid X. $$ The conditional statement means that, within every covariate stratum $X=x$ under consideration, the joint distribution of the potential outcomes and potential treatment receipts is the same for $Z=1$ and $Z=0$.

This is a claim about counterfactual quantities, most of which cannot be observed for any one unit. It is stronger than saying that the observed groups have similar average outcomes, similar treatment rates, or similar measured characteristics. It is also distinct from instrument relevance: relevance concerns whether $Z$ changes the distribution of $D$. Independence concerns whether the instrument groups differ in their underlying potential responses.

Measured Common Causes and Conditional Independence

A measured common cause is a pre-instrument variable that helps determine both instrument receipt and potential outcomes or treatment receipt. For instance, a clinician may be more likely to make a referral for a patient with severe symptoms, while severity also predicts what that patient’s outcome would be under either treatment condition. In that situation, referral status may be associated with potential outcomes because the referral groups differ in severity.

If the relevant common cause is measured, conditioning on it may make an observational independence assumption more plausible. The claim then becomes that, among people with the same measured covariates, instrument receipt does not further sort people by their potential outcomes or treatment responses. This requires more than including covariates in a regression: the covariates must be measured appropriately, precede instrument assignment, and capture the common causes relevant to the assignment mechanism and potential responses. Conditioning on measured covariates does not, by itself, remove differences due to unmeasured common causes.

The conditional assumption is often paired with a support condition. For covariate values where an IV contrast is to be estimated, both instrument values must be possible:

0 < P(Z=1\mid X=x) < 1.

This overlap condition is not a substitute for independence. It says that the data contain instrument variation within the covariate stratum; it does not say that the instrument is independent of potential outcomes there.

Worked Example: Referral Instrument and Baseline Severity

Suppose, hypothetically, that a clinic’s referral recommendation is used as an instrument for receiving a treatment. Let $Z=1$ denote a recommendation and $D=1$ denote treatment receipt. Baseline severity is measured before the recommendation. Assume the population has two severity groups, low and high, and that within each group the recommendation is independent of the joint set of potential outcomes and treatment receipts.

Suppose 80% of the recommended group has high severity, compared with 20% of the non-recommended group. For illustration, let the mean potential outcome under no treatment be 4 units for low-severity patients and 8 units for high-severity patients. Then:

E[Y(0)\mid Z=1]=0.20(4)+0.80(8)=7.20, $$ $$ E[Y(0)\mid Z=0]=0.80(4)+0.20(8)=4.80.

The potential-outcome means differ by $7.20-4.80=2.40$ units across instrument groups. Thus, unconditional independence fails in this hypothetical population, even though assignment is independent of the potential quantities within each severity group by construction. Severity explains the difference in group composition.

This calculation illustrates why simply comparing the instrument groups can be misleading. It does not establish that conditioning on measured severity will make independence credible in a real clinic. That conclusion would also require that severity captures the relevant common causes and that recommendation is not still related to potential responses within severity groups.

What Diagnostics Can Show

Observed covariate balance is useful for checking whether the instrument groups differ on measured, pre-instrument characteristics. A difference on a strong prognostic variable is a warning that unconditional independence may be implausible and that a conditional analysis may be needed. Balance after stratification, matching, or adjustment can show that the observed distribution of those measured variables has been made more comparable. None of these checks observes the full joint distribution of $Y(0),Y(1),D(0),D(1)$.

A baseline outcome or prognostic score measured before the instrument can also be informative: a difference across instrument groups may signal differences in underlying prognosis. But a small or absent difference on such a measure is not proof of independence. It only addresses the particular observed measure, and it cannot guarantee balance on unmeasured predictors or on treatment responses that are never jointly observed.

Worked Example: Standardizing a Measured Prognostic Score

Imagine a hypothetical study with a binary instrument and a pre-instrument prognostic score. The score averages 2 points in the low-risk group and 8 points in the high-risk group. Among those with $Z=1$, 80% are high-risk; among those with $Z=0$, 20% are high-risk. Within each risk group, suppose the score means are the same for both instrument values.

The unadjusted group means are:

\bar{x}_{Z=1}=0.20(2)+0.80(8)=6.80, $$ $$ \bar{x}_{Z=0}=0.80(2)+0.20(8)=3.20.

The unadjusted difference is $6.80-3.20=3.60$ score points. Now standardize both instrument groups to a hypothetical target distribution that is 50% low-risk and 50% high-risk:

\bar{x}_{Z=1}^{\,\mathrm{std}}=0.50(2)+0.50(8)=5.00, $$ $$ \bar{x}_{Z=0}^{\,\mathrm{std}}=0.50(2)+0.50(8)=5.00.

The standardized difference is $5.00-5.00=0.00$ points. This shows that the measured score difference in this example is explained by the two groups’ different risk composition. It is a diagnostic about one measured variable, not evidence that the complete conditional independence assumption holds. In particular, the arithmetic does not test whether the groups have the same unobserved potential treatment responses.

A Practical Assessment

A defensible assessment begins with the actual process that generated $Z$, not with a statistical test of whether the observed instrument–outcome association is significant. Describe why units received different instrument values, identify pre-instrument variables that could affect both that process and potential responses, and state which of those variables were measured. Then assess whether there is instrument variation within the covariate strata used for the analysis.

For example, suppose a hypothetical education study uses distance to a college as an instrument for college attendance. Family resources, local school quality, and urban or rural location might affect distance while also predicting potential earnings or college attendance under different distance conditions. Measuring such characteristics can motivate a conditional independence argument, but only if the remaining within-stratum differences in distance are plausibly unrelated to the full set of potential outcomes and treatment receipts. A list of measured covariates is not enough: the substantive assignment mechanism still matters.

Describe instrument receipt.
Explain how units came to have $Z=1$ or $Z=0$, using the setting’s actual assignment process.

Identify pre-instrument common causes.
List measured characteristics that may influence both instrument receipt and potential outcomes or treatment receipt. Distinguish these from variables measured after the instrument.

Check covariate balance and overlap.
Examine measured differences and whether both instrument values occur within the covariate strata used. Treat these as diagnostics, not tests that prove independence.

State the remaining assumption.
Explain why, after conditioning on the chosen $X$, instrument receipt is plausibly unrelated to the joint potential quantities. Acknowledge what unmeasured differences could remain.

Why the Conditional Claim Helps—and Where It Stops

The justification for conditioning can be stated directly. Let $V=\{Y(0),Y(1),D(0),D(1)\}$. If $Z\perp V\mid X$, then for any function $g$ for which the expectation exists, and for each relevant $x$,

E[g(V)\mid Z=1,X=x]=E[g(V)\mid Z=0,X=x]=E[g(V)\mid X=x].

Thus, within a covariate stratum, the two instrument groups have the same distribution of the potential quantities, not just the same mean for one selected quantity. This is why a conditional IV analysis can be justified when unconditional independence is implausible but the conditional claim is defensible. The argument relies on the independence assumption; it does not derive that assumption from observed balance.

Several limits follow. First, balance checks only measured covariates, while independence concerns counterfactual quantities that are largely unobserved. Second, even if conditional independence is credible, lack of overlap means the data do not support comparing instrument values in some covariate strata. Third, covariates should be chosen using knowledge of the pre-instrument process; indiscriminate adjustment does not turn an invalid instrument into a valid one. The earlier material on conditional independence given covariates provides the formal role of $X$ in IV identification; the central task here is judging whether that condition is credible in an observational setting.

Comparing Independence Strategies

Approach	What supports the claim	Main limitation
Unconditional independence	A design or assignment process makes $Z$ unrelated to potential responses across the population.	Often difficult to defend when instrument receipt is selected in observational data.
Conditional independence	Measured pre-instrument covariates account for the relevant common causes, with instrument variation remaining within strata.	Unmeasured common causes can remain; covariate balance alone cannot establish the assumption.
Randomized instrument assignment	Known random assignment provides a design-based reason for independence, subject to the design being carried out as specified.	May not be available; random assignment does not itself establish other IV assumptions.
Direct treatment adjustment	Uses measured treatment–outcome confounders under a treatment-exchangeability assumption rather than relying on an instrument.	It is a different identification strategy and also depends on assumptions about measured and unmeasured confounding.

Common Mistake

Wrong reasoning: “After adjusting for the measured covariates, the instrument groups have similar baseline scores, so the instrument is independent of potential outcomes and treatment receipt.”

Correction: Similarity on measured baseline scores supports only a limited diagnostic claim. Independence concerns the joint distribution of all potential outcomes and treatment receipts. It requires a substantive argument that the measured covariates capture the relevant common causes, as well as adequate overlap. An observational analysis cannot verify that claim merely by finding small measured differences or a nonsignificant balance test.

Key takeaway: In observational IV analysis, conditioning on measured pre-instrument common causes can make independence more plausible, but it does not prove independence. Explain the assignment process, assess balance and overlap, and state explicitly what unmeasured differences could still undermine the claim.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In an observational IV study, a measured baseline severity variable predicts both instrument receipt and prognosis. Which independence condition is most relevant if the analysis conditions on severity?
A covariate has a mean of 10 in the $Z=1$ group and 10 in the $Z=0$ group. Does this establish instrument independence? Why or why not?
Within a covariate stratum, every unit has $Z=1$. What condition is missing for estimating a within-stratum instrument contrast?
Why does a difference in observed outcomes between instrument groups fail, by itself, to demonstrate that independence is violated?

Solutions. (1) The relevant claim is $Z\perp\{Y(0),Y(1),D(0),D(1)\}\mid X$, where severity is included in $X$. It must be defended within severity strata, not inferred from adjustment alone. (2) No. Equal means for one measured covariate do not establish equality of its full distribution, balance on other variables, or independence of the unobserved potential quantities. (3) Overlap is missing: both $Z=1$ and $Z=0$ must be possible in the stratum, so $0<P(Z=1\mid X=x)<1$. (4) Observed outcomes combine instrument-related treatment variation with differences in group composition and other features of the data. A group difference is not a direct comparison of both groups’ potential outcomes under the same treatment condition, so it cannot alone diagnose the independence assumption.

Approach	What supports the claim	Main limitation
Unconditional independence	A design or assignment process makes \(Z\) unrelated to potential responses across the population.	Often difficult to defend when instrument receipt is selected in observational data.
Conditional independence	Measured pre-instrument covariates account for the relevant common causes, with instrument variation remaining within strata.	Unmeasured common causes can remain; covariate balance alone cannot establish the assumption.
Randomized instrument assignment	Known random assignment provides a design-based reason for independence, subject to the design being carried out as specified.	May not be available; random assignment does not itself establish other IV assumptions.
Direct treatment adjustment	Uses measured treatment–outcome confounders under a treatment-exchangeability assumption rather than relying on an instrument.	It is a different identification strategy and also depends on assumptions about measured and unmeasured confounding.