Why Independence Needs Special Attention With Observational Instruments
An instrument is useful because it creates variation in treatment that can support a causal comparison. In a randomized encouragement study, the assignment mechanism may provide a direct reason to regard that variation as unrelated to participants’ potential outcomes and treatment choices. For an observational instrument, such as distance to a facility or assignment to a particular institution, the instrument is not randomized by design. Its apparent usefulness for predicting treatment does not by itself explain why its values are unrelated to the factors that shape outcomes.
The key task is therefore to assess the assignment mechanism, not just the instrument’s association with treatment. Ask how instrument values arise, which people or places receive different values, and whether those differences could also be related to prognosis or treatment response. The assumption concerns potential outcomes, so it cannot be verified directly from the observed data.
The joint form matters. Independence from \(Y(0)\) alone would not rule out a relationship with the outcome under treatment, and independence from the outcome potentials alone would not rule out sorting by treatment response. Including \(D(0)\) and \(D(1)\) means the instrument is also assumed unrelated to which treatment choice a person would make under either instrument value.
Independence is one IV assumption, not a replacement for the others. An instrument can be independent yet fail to predict treatment; it can predict treatment yet be related to potential outcomes. Independence also does not establish the exclusion restriction, which concerns how the instrument can affect the outcome. These questions must be considered separately.
What Independence Implies—and What It Does Not
If the independence assumption holds, then the distribution of the potential-outcome and potential-treatment vector is the same across instrument groups. In particular, the potential outcome under either fixed treatment level has the same mean in the two groups, as does the potential treatment under either instrument value:
These are implications, not a way to calculate the unobserved quantities: for each person, only one instrument value and one treatment are observed. For example, the data do not reveal both \(Y(0)\) and \(Y(1)\) for the same person. Independence therefore cannot be checked by simply comparing observed outcomes across instrument groups. Observed outcomes can differ because the instrument changes treatment, even when independence is true.
Observed baseline covariates can still help assess whether the assignment story is plausible. If a measured covariate is a strong predictor of potential outcomes and differs substantially across instrument groups, that is a warning sign. But independence does not automatically require balance on every recorded variable: some variables may be unrelated to the potential quantities, and finite samples can show chance imbalances even under random assignment. Balance checks are diagnostic evidence, not proof of independence.
Worked Examples: Assessing Assignment Mechanisms
Worked Example: Distance to a Treatment Facility
Suppose a hypothetical study uses whether a patient lives within 10 kilometres of a specialised clinic as an instrument for receiving a particular treatment. Researchers compare 200 patients in each distance group. Age is a pre-treatment predictor of the outcome. The nearby group has mean age 52 years and standard deviation 11 years; the farther group has mean age 59 years and standard deviation 12 years. Assess this balance diagnostic.
The mean difference, defined as nearby minus farther, is $$ 52-59=-7\text{ years}. $$ The pooled standard deviation is $$ s_{\text{pooled}}=\sqrt{\frac{(200-1)(11^2)+(200-1)(12^2)}{200+200-2}} =\sqrt{\frac{199(121)+199(144)}{398}} =\sqrt{132.5} \approx 11.51\text{ years}. $$
The standardized mean difference is $$ \frac{-7}{11.51}\approx -0.608. $$ Thus, the nearby group is about 0.61 pooled standard deviations younger on average in this hypothetical sample. That is a notable imbalance in a prognostic variable and gives reason to question whether the instrument groups have comparable potential outcomes.
The calculation does not prove that independence fails: age is observed, while the assumption concerns potential quantities, and other aspects of the assignment mechanism matter. Still, distance may also reflect income, urbanicity, travel access, or local disease burden. Those are concrete routes by which instrument values could be associated with prognosis. Even if distance strongly predicts treatment, the first-stage association does not remove these concerns.
Worked Example: An Eligibility Rule Based on Income
A hypothetical programme uses income below a specified threshold as an instrument for receiving a support service. In the full study population, the mean annual income is $28,000 among those classified as eligible and $46,000 among those classified as ineligible. Suppose income also predicts the outcome, perhaps because it is related to housing stability and access to care. Assess the independence claim for this full-population comparison.
The observed income difference, eligible minus ineligible, is $$ \$28{,}000-\$46{,}000=-\$18{,}000. $$ Eligibility is determined by income, so the instrument groups differ by construction on a variable that is also prognostic. This makes a claim of global independence implausible: the assignment rule directly sorts people using a potential source of outcome differences.
The rule may still predict receipt of the service, but prediction is relevance, not independence. Nor does naming a threshold make assignment effectively random throughout the population. A separate design argument would be needed to justify a comparison in a narrower setting; this example does not establish such an argument. For the full-population instrument defined here, the documented income-based sorting is a direct warning against treating eligibility as independent of potential outcomes.
Worked Example: Assignment by an Institutional Rotation
A hypothetical hospital rotates patients between two clinical teams, and team assignment is proposed as an instrument for receiving a procedure. The rotation is described as administrative. However, the first team sees 65% weekday admissions, while the second sees 35% weekday admissions. Assume weekday admissions tend to involve different case severity and that severity predicts outcomes.
The weekday-admission proportion difference, first team minus second team, is $$ 0.65-0.35=0.30, $$ or 30 percentage points. The assignment groups are therefore not balanced on this measured feature of the admission process. If weekday timing is linked to case severity, this imbalance raises a specific concern that team assignment may be related to potential outcomes.
Calling a rotation “administrative” is not enough to establish independence. The relevant questions include whether the rotation was followed, whether staff could influence assignment, whether patients were sorted into admission times, and whether team assignment coincided with other institutional differences. If the rotation were genuinely implemented without regard to patient prognosis and no related sorting occurred, that could support the assumption; the weekday imbalance gives reason to investigate rather than assume that it did.
A Practical Assessment of an Independence Claim
For an observational instrument, a defensible assessment starts with a concrete account of how \(Z\) is assigned. The account should explain why people with different instrument values would have comparable potential outcomes and treatment choices, apart from the effect of the instrument on treatment. A vague statement that the instrument is “external” or “administrative” does not supply that explanation.
Define the instrument and its values precisely. For distance, state the measure and comparison; for eligibility, state the rule; for institutional assignment, describe the actual allocation process.
Ask who or what determines instrument values, whether people can sort into them, and whether the rule changes with location, time, staff, resources, or patient characteristics.
Consider whether the same causes of instrument assignment could predict either potential outcome or which treatment a person would take under each instrument value.
Compare relevant pre-treatment predictors and examine the assignment records. Imbalances or sorting patterns can weaken the case; their absence cannot establish independence.
This assessment distinguishes an assumption’s plausibility from its testability. If instrument assignment is documented as random within a well-defined mechanism, that can provide a strong design-based justification. With an observational instrument, the justification is usually substantive: the assignment process and available evidence must make independence credible, while acknowledging remaining uncertainty.
Common Mistake: Treating a Balance Test as Proof
A common but incorrect argument is: “The instrument groups have no statistically significant difference in baseline covariates, so independence is established.” A nonsignificant test means the data did not provide enough evidence against the tested equality; it does not show the covariates are equal. The test also concerns only the measured variables and cannot reveal imbalance in unmeasured causes or directly compare unobserved potential outcomes.
The correction is to treat balance as one diagnostic among several. Report meaningful differences and their context, examine how the instrument was assigned, and explain why plausible common causes of instrument values and potential outcomes are absent or limited. Do not convert a failure to detect imbalance into evidence that all potential outcomes are independent of the instrument.
Check Your Understanding
For each question, distinguish evidence about the assignment process from what the independence assumption itself asserts.
- State the independence assumption using \(Z\), \(D(z)\), and \(Y(d)\). Why does it include potential treatment choices as well as potential outcomes?
- A distance instrument predicts treatment strongly, but distance is associated with neighbourhood income. Why does the strong treatment association not resolve the independence concern?
- An eligibility rule assigns instrument values directly from a prognostic variable. What does that imply for a global independence claim in the population defined by the rule?
- Why can a nonsignificant baseline-balance test not establish independence from potential outcomes?
- Give one feature of an institutional assignment mechanism that would support an independence argument and one feature that would undermine it.