What Data Can and Cannot Establish About IV Assumptions

Observed Patterns and IV Assumptions

Instrumental-variable analyses rely on assumptions about how an instrument $Z$ relates to treatment receipt $D$ and outcome $Y$. Some features of that relationship can be examined in observed data. Other assumptions concern potential outcomes or treatment choices that are never jointly observed for the same person. Keeping these categories separate matters: a dataset can reveal a nonzero instrument–treatment association without demonstrating that the association supports a causal interpretation.

For clarity, consider a binary instrument $Z\in\{0,1\}$, a binary treatment $D\in\{0,1\}$, and an outcome $Y$. Let $D(z)$ denote the treatment a person would receive if their instrument were set to $z$, and let $Y(d)$ denote their outcome under treatment status $d$. Under consistency, the observed values satisfy $D=D(Z)$ and $Y=Y(D)$. The four assumptions discussed here are relevance, independence, exclusion, and monotonicity.

Definition: Relevance requires a nonzero population treatment-receipt contrast across instrument values: $E[D\mid Z=1]\ne E[D\mid Z=0]$. Independence requires $Z$ to be independent of the potential treatment choices and potential outcomes, $(D(0),D(1),Y(0),Y(1))\perp Z$. Exclusion requires the instrument to affect the outcome only through treatment, so the outcome under instrument value $z$ and treatment $d$ is $Y(d)$, not a separate function of $z$. Monotonicity requires $D(1)\ge D(0)$ for every person, ruling out defiers.

The relevance condition is stated for a population quantity. In a sample, the observed treatment-receipt contrast is $\hat p_1-\hat p_0$, where $\hat p_z$ is the fraction receiving treatment among those with $Z=z$. This contrast is directly calculable. Its uncertainty can be assessed, but a nonzero sample contrast alone does not prove a nonzero population contrast. Even strong evidence of relevance addresses only the instrument–treatment link; it does not establish independence, exclusion, or monotonicity.

What a Treatment-Receipt Contrast Shows

A nonzero contrast in observed treatment receipt is evidence relevant to the first-stage condition. Its interpretation depends on how $Z$ was assigned and on sampling uncertainty. If the instrument was randomized, random assignment supports independence by design, but the observed contrast still varies across samples. In an observational setting, the same contrast may reflect a relationship between the instrument and treatment receipt, but it does not by itself show why that relationship exists or whether the other assumptions hold.

Worked Example: A Randomized Encouragement With a Nonzero First Stage

Suppose a hypothetical encouragement trial assigns 100 people to $Z=1$ and 100 to $Z=0$. Sixty people in the encouragement group and 30 in the other group receive treatment. The observed treatment-receipt rates are $0.60$ and $0.30$, respectively.

Calculate the contrast. The observed treatment-receipt contrast is

\hat p_1-\hat p_0=\frac{60}{100}-\frac{30}{100}=0.60-0.30=0.30.

Thus the observed contrast is 30 percentage points, not zero. It informs relevance: the sample shows an association between assignment and treatment receipt. It does not establish independence, exclusion, or monotonicity.

For an approximate standard error of the difference in two independent sample proportions, use

SE=\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_0(1-\hat p_0)}{n_0}} =\sqrt{\frac{0.60(0.40)}{100}+\frac{0.30(0.70)}{100}} =\sqrt{0.0045}\approx 0.0671.

A normal-approximation 95% confidence interval for the population contrast is $0.30\pm1.96(0.0671)$, or approximately $[0.169,0.431]$. Under the sampling assumptions for this interval, the data provide evidence that the population contrast is positive. The interval concerns relevance, not the validity of the other IV assumptions. Random assignment supports independence through the design; the treatment and outcome counts alone cannot prove exclusion or individual-level monotonicity.

The treatment contrast also has a useful potential-outcome interpretation under independence. For binary treatment, people fall into four response types: always-takers have $D(1)=D(0)=1$, never-takers have $D(1)=D(0)=0$, compliers have $D(1)=1,D(0)=0$, and defiers have $D(1)=0,D(0)=1$. Without monotonicity, both compliers and defiers may exist.

Justification: If independence holds, the observed treatment means at each instrument value equal the corresponding population means of the potential treatment choices. Therefore,

E[D\mid Z=1]-E[D\mid Z=0] =E[D(1)]-E[D(0)] =E[D(1)-D(0)] =P(\text{complier})-P(\text{defier}).

The final equality follows because $D(1)-D(0)$ equals $1$ for a complier, $-1$ for a defier, and $0$ for an always-taker or never-taker. Under monotonicity, the defier proportion is zero, so the population contrast equals the complier proportion. But a positive contrast without monotonicity only indicates that compliers outnumber defiers; it does not show there are no defiers.

Assumptions That Observed Data Cannot Generally Verify

Independence is a claim about the relationship between the instrument and potential responses, including responses that are unobserved for each person. In a randomized instrument design, the assignment mechanism provides a basis for believing that $Z$ is independent of those potential responses. In an observational design, researchers can compare measured pre-instrument characteristics across instrument groups. Such comparisons can reveal imbalance and raise concerns, but balance on recorded variables cannot establish independence from unmeasured characteristics or potential outcomes.

Worked Example: Measured Balance Does Not Prove Independence

In a hypothetical policy study, eligibility $Z$ is associated with treatment receipt: 55% of eligible people receive treatment, compared with 35% of ineligible people. The observed contrast is $0.55-0.35=0.20$, or 20 percentage points. The sample also shows that eligible people have a mean baseline income of \$48,000, while ineligible people have a mean of \$45,000.

The treatment contrast is observable evidence relevant to relevance, subject to sampling uncertainty. The income difference is an observable imbalance that may be a warning sign: if income predicts treatment choices or potential outcomes, it may indicate a threat to independence. But the comparison does not by itself prove that independence fails, nor could equal observed means prove independence. The potential outcomes and any unmeasured determinants are not fully observed. A researcher must use knowledge of the assignment mechanism and substantive context, not just a balance table, to assess this assumption.

Exclusion is likewise not generally testable from the observed distribution alone. For each person, only one instrument value and one resulting outcome are observed. A difference in mean outcomes across instrument groups could arise because the instrument changes treatment receipt and treatment changes outcomes, because the instrument has a direct effect, or because the groups differ in potential outcomes. Conversely, a zero outcome contrast does not prove exclusion: treatment effects and direct effects could offset one another. Observed outcome patterns are informative, but they do not reveal the missing counterfactual outcomes needed to isolate a direct effect.

Worked Example: An Outcome Contrast Does Not Diagnose Exclusion

Return to the hypothetical encouragement trial. Suppose mean outcomes are 18 units for $Z=1$ and 15 units for $Z=0$. The observed outcome contrast is $18-15=3$ units, while the treatment-receipt contrast is $0.30$. Dividing these contrasts gives $3/0.30=10$ outcome units per unit change in treatment receipt.

That calculation is arithmetic, not a test of exclusion. If independence, exclusion, and monotonicity hold, the ratio has a causal interpretation as the average treatment effect among compliers. But the observed contrasts do not demonstrate those assumptions. For example, a direct effect of encouragement could contribute to the 3-unit outcome contrast; alternatively, a difference in potential outcomes between instrument groups could contribute if independence fails. The same observed means are therefore compatible with different causal explanations. The appropriate conclusion from these data alone is that the instrument groups differ by 3 outcome units and 30 percentage points in treatment receipt, not that exclusion has been verified.

Monotonicity is an individual-level restriction on two potential treatment choices. Each person’s observed record reveals only $D(1)$ or $D(0)$, not both. A nonzero treatment contrast therefore cannot reveal whether any individual is a defier.

Worked Example: The Same First Stage With and Without Defiers

Consider a hypothetical population in which 50% are never-takers, 30% are compliers, 10% are defiers, and 10% are always-takers. With $Z=0$, the treatment rate is the proportion of always-takers plus defiers: $0.10+0.10=0.20$. With $Z=1$, it is the proportion of always-takers plus compliers: $0.10+0.30=0.40$. The treatment-receipt contrast is $0.40-0.20=0.20$.

Now consider a different hypothetical population with 60% never-takers, 20% compliers, no defiers, and 20% always-takers. Its treatment rates are $0.20$ at $Z=0$ and $0.40$ at $Z=1$, giving the same observed contrast of 0.20. Thus the treatment-rate comparison cannot distinguish these populations. A positive first stage is consistent with monotonicity, but it does not establish it.

Diagnostics, Alternatives, and Limits

Observed diagnostics are useful when their evidential scope is stated carefully. Treatment-receipt contrasts assess the empirical first stage; uncertainty intervals indicate how precisely it is estimated. In observational settings, comparisons of pre-instrument covariates can expose differences that challenge a proposed independence argument. Outcome contrasts describe what happened across instrument groups, but they do not by themselves distinguish treatment-mediated effects from direct effects or confounding. No single diagnostic substitutes for a defensible assignment mechanism and a substantive argument for exclusion and monotonicity.

Assumption	Observable information	What that information cannot establish alone
Relevance	Instrument-specific treatment rates and their contrast	That the population contrast is nonzero with certainty, or that other assumptions hold
Independence	Measured baseline balance and information about assignment	Independence from unmeasured factors and all potential responses
Exclusion	Observed outcome and treatment contrasts across instrument groups	That the instrument affects outcomes only through treatment
Monotonicity	Aggregate treatment rates by instrument value	That no individual would move against the instrument’s direction

The main alternative to relying on an observational instrument is a design that randomizes the instrument; randomization directly strengthens the independence argument. It does not, by itself, guarantee exclusion or monotonicity. Covariate adjustment and balance checks can address measured differences but cannot certify independence. These approaches provide different kinds of support rather than a universal empirical test of IV validity.

A common mistake is to reason, “Treatment receipt is higher when $Z=1$, so the instrument is valid.” The correction is that this observed nonzero treatment contrast informs relevance only. It does not establish independence, exclusion, or monotonicity, and sampling uncertainty means the sample contrast is not identical to a known population contrast. A careful IV argument separates what the data show from what the design and substantive knowledge must justify.

Key takeaway: Data can quantify the instrument–treatment association and reveal observable patterns that raise concerns. They generally cannot verify assumptions defined using unobserved potential outcomes or individual counterfactual treatment choices. Report the observable evidence, then state separately the design and substantive grounds for each assumption.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A sample has treatment rates 0.42 for $Z=1$ and 0.27 for $Z=0$. What is the observed treatment-receipt contrast, and which assumption does it inform?
Why does equal mean age across instrument groups not prove independence?
Can a zero observed outcome contrast establish exclusion? Explain.
Under independence, a population has a complier proportion of 0.25 and a defier proportion of 0.05. What is the population first-stage contrast, and does it establish monotonicity?

Solutions. (1) The contrast is $0.42-0.27=0.15$, or 15 percentage points. It informs relevance; it does not establish the other assumptions. (2) Equal means address only one measured characteristic and do not rule out differences in unmeasured characteristics or potential responses. (3) No. A zero contrast could occur despite a direct effect, including when effects offset; the observed outcomes do not reveal the necessary counterfactuals. (4) The contrast is $0.25-0.05=0.20$, or 20 percentage points. It is positive, but the positive defier proportion shows that monotonicity fails.