Why Separate Assumptions From Their Observable Implications?
An instrumental-variable analysis rests on claims about how an instrument \(Z\) relates to treatment and outcomes. Some features of a study can be examined directly in the observed data: for example, whether treatment uptake differs across instrument groups, or whether measured baseline characteristics appear balanced. Other assumptions concern counterfactual quantities that are never jointly observed for a person. A diagnostic can be informative without being a test that establishes those assumptions.
This distinction matters in both directions. A large observed first stage is evidence that the instrument is relevant in the sample, but it does not establish independence or exclusion. Conversely, a lack of statistically detectable imbalance in measured covariates does not show that the instrument is independent of unmeasured causes of treatment and outcome.
What the Assumptions Say
For a binary instrument, write \(Z=1\) for one assignment or encouragement and \(Z=0\) for the other. Let \(D\) be a binary treatment, \(D(z)\) the treatment a person would receive if assigned instrument value \(z\), and \(Y(d)\) the outcome that person would have under treatment value \(d\). The observed treatment is \(D=D(Z)\). Under exclusion, the observed outcome is \(Y=Y(D)\): the instrument has no effect on the outcome except through treatment.
These assumptions refer to potential outcomes or potential treatments. The observed data contain only the assigned \(Z\), the resulting \(D\), and the resulting \(Y\) for each person. In particular, a person is observed under only one instrument value, so the other potential treatment \(D(1-Z)\) is not observed. The other potential outcome \(Y(1-D)\) is also not observed.
A baseline covariate \(X\) needs careful treatment in balance checks. The independence statement above does not by itself imply \(Z\perp X\) for an arbitrary \(X\). If the assignment claim includes \(X\), so that \(\{D(0),D(1),Y(0),Y(1),X\}\perp Z\), then it does imply balance in \(X\). Random assignment, when properly implemented, is one design-based reason to expect instrument groups to be balanced on baseline characteristics. Whether \(X\) belongs in the independence claim must be made explicit.
Observable Checks: Relevance and Balance
The first-stage contrast is calculated from observed treatment and instrument values. It describes how treatment uptake differs between the two instrument groups. A nonzero sample contrast is directly visible; whether it provides strong evidence of a nonzero population contrast depends on sampling uncertainty and the study design. A contrast near zero may indicate that the instrument is weakly related to treatment, but a sample contrast alone is not proof of population relevance.
For baseline characteristics, one can compare distributions across instrument groups: means for quantitative variables, proportions for binary variables, or category frequencies for categorical variables. These comparisons are useful diagnostics when the design or assignment assumption entails balance. A discrepancy can prompt investigation of assignment procedures, data errors, or chance variation. A comparison of measured covariates cannot directly assess balance on unmeasured characteristics.
Worked Example: Measuring the Observed First Stage
In a hypothetical encouragement study, 500 people are assigned \(Z=1\) and 500 are assigned \(Z=0\). Treatment is received by 310 people in the \(Z=1\) group and 250 people in the \(Z=0\) group. Calculate and interpret the sample first-stage contrast.
The observed treatment proportions are \(310/500=0.620\) for \(Z=1\), and \(250/500=0.500\) for \(Z=0\). Therefore,
In this hypothetical sample, treatment receipt is 12.0 percentage points higher in the \(Z=1\) group. This is an observable first-stage contrast. If \(Z\) was randomly assigned, the contrast can also be interpreted as an estimate of the assignment effect on treatment uptake, subject to sampling uncertainty. It does not establish that the instrument is independent of all potential outcomes, nor does it establish exclusion.
Worked Example: A Baseline Covariate Balance Check
Suppose the same hypothetical study records baseline smoking status. Of 500 people assigned \(Z=1\), 150 were smokers at baseline; of 500 assigned \(Z=0\), 170 were smokers. Calculate the observed difference in smoking prevalence, taking the \(Z=1\) group minus the \(Z=0\) group.
The sample proportions are \(150/500=0.300\) and \(170/500=0.340\). Thus the difference is
The estimated smoking prevalence is 4.0 percentage points lower in the \(Z=1\) group. The negative sign matters: the first group’s observed proportion is lower, not higher.
For illustration, treating the two groups as independent samples, the usual estimated standard error for this difference in proportions is
The observed difference is about \(-0.040/0.0295=-1.36\) standard errors from zero. This is not a large discrepancy relative to that sampling standard error. It is still only a check of one measured baseline characteristic, not proof that the groups are alike in every relevant respect. In a randomized design, small imbalances can occur by chance; across many covariates, some differences can also arise by chance alone.
Why Balance Follows Only From a Broader Assignment Claim
The implication for a covariate can be justified directly. If the assignment assumption explicitly includes \(X\), it states that \(X\) and \(Z\) are independent. Independence means that the distribution of \(X\) is the same conditional on either instrument value (when both values have positive probability). For a binary \(X\), this gives the same prevalence in the two groups in the population.
The conclusion does not follow if \(X\) was omitted from the independence assumption. For example, suppose all the potential treatments and outcomes are constant, but \(X=Z\). Then \(Z\) is independent of the potential treatments and outcomes, because constants do not vary with \(Z\); nevertheless, \(X\) is completely unbalanced across instrument groups. Thus a balance check has a clear interpretation only in relation to an assignment claim that includes the covariate being checked.
Even when the assignment assumption includes measured covariates, observed balance is a diagnostic rather than a verification of independence from the full collection of potential outcomes. Measured covariates can be balanced while unmeasured variables differ. And in a randomized study, an observed covariate difference does not, by itself, logically disprove random assignment; it may be an unlikely or ordinary chance realization, depending on its size and the design.
Why Exclusion and Monotonicity Are Not Generally Testable
Exclusion is a claim about what would happen under instrument and treatment combinations not necessarily observed. The data show \(Y\) under the treatment actually received and the instrument actually assigned. They do not show, for the same person, the outcome under the same treatment but a different instrument value. Consequently, the absence of an obvious instrument-to-outcome pathway in observed data is not a direct test of exclusion. Nor is an association between \(Z\) and \(Y\) by itself evidence against exclusion: if the instrument changes treatment and treatment changes outcome, an association between instrument and outcome is possible even when exclusion holds.
Monotonicity is also a statement about unobserved counterfactuals. It rules out a person who would take treatment under \(Z=0\) but not under \(Z=1\). Since each person experiences only one instrument value, the data reveal either \(D(0)\) or \(D(1)\), not both. A difference in average uptake across groups cannot show that every individual moves in the same direction.
Worked Example: The Same Observed Data Under Different Exclusion Claims
Consider a hypothetical randomized study with 100 people in each instrument group. The observed data have 40 treated people in the \(Z=0\) group and 60 in the \(Z=1\) group; for every person, the observed outcome equals the observed treatment. Thus, in each group, treated people have \(Y=1\) and untreated people have \(Y=0\).
One causal model consistent with these data has 40% always-takers, 20% compliers, and 40% never-takers. Their potential treatments are, respectively, \((D(0),D(1))=(1,1),(0,1),(0,0)\). Let their outcomes satisfy \(Y(d,z)=d\). This model has exclusion: at a fixed treatment value, changing \(z\) does not change the outcome. It generates treatment rates of \(0.40\) and \(0.60\), and observed outcomes equal observed treatment.
A second model uses the same treatment types and proportions, so it generates exactly the same observed treatment data. Define its outcomes as follows: when \(d=D(z)\), set \(Y(d,z)=D(z)\); when \(d\ne D(z)\), set \(Y(d,z)=z\). In the observed data, \(d=D(z)\) always, so this model also gives \(Y=D\) for every person and reproduces the same observed table. But for a complier, \(D(0)=0\) and \(D(1)=1\). At treatment \(d=0\), this model gives \(Y(0,0)=0\) and \(Y(0,1)=1\), so exclusion fails.
The two models agree on the observed data but differ on exclusion. The example shows why observing a particular relationship among \(Z\), \(D\), and \(Y\) cannot, by itself, determine whether exclusion holds. The unobserved counterfactual outcomes distinguish the models.
Worked Example: Uptake Rates Do Not Reveal Every Compliance Type
Suppose a hypothetical study has population treatment rates \(P(D=1\mid Z=0)=0.40\) and \(P(D=1\mid Z=1)=0.60\). The first-stage difference is \(0.60-0.40=0.20\). Compare two possible populations of potential treatment types, listed as percentages.
| Population of types | Always-takers | Compliers | Defiers | Never-takers | Rate under \(Z=0\) | Rate under \(Z=1\) |
|---|---|---|---|---|---|---|
| Population A | 40% | 20% | 0% | 40% | 40% | 60% |
| Population B | 30% | 30% | 10% | 30% | 40% | 60% |
In Population A, the treatment rate under \(Z=0\) is \(40\%+0\%=40\%\), from always-takers and defiers; under \(Z=1\), it is \(40\%+20\%=60\%\), from always-takers and compliers. There are no defiers, so monotonicity holds.
In Population B, the rate under \(Z=0\) is \(30\%+10\%=40\%\), and the rate under \(Z=1\) is \(30\%+30\%=60\%\). This population contains 10% defiers, so monotonicity fails. The same two uptake rates are therefore compatible with both a population satisfying monotonicity and one violating it. The observed first stage does not reveal the joint distribution of \(D(0)\) and \(D(1)\) for individuals.
A Practical Classification
| Claim or feature | What is observed or checked? | What the check can establish |
|---|---|---|
| Relevance | Compare treatment uptake across instrument groups. | Describes the sample first stage; inference can assess evidence about a population contrast. |
| Balance in measured \(X\) | Compare baseline covariate distributions by instrument group. | Diagnoses measured imbalance when the assignment claim includes \(X\); does not establish balance on unmeasured variables. |
| Independence from potential treatments and outcomes | Use design information and measured balance as diagnostics. | Cannot be verified in full from observed outcomes and treatments, because potential quantities are missing. |
| Exclusion | Consider observed associations and the study’s causal structure. | Cannot generally be tested from the observed distribution alone; the relevant counterfactual outcomes are unobserved. |
| Monotonicity | Observe treatment under each person’s assigned instrument value. | Cannot directly establish that no person would respond in the opposite direction, because both potential treatments are not observed. |
The most useful interpretation is therefore not “the assumptions were tested and passed.” It is more precise to report the observed checks, state which assumptions those checks diagnose, and explain what remains dependent on design knowledge or substantive judgment. Assignment procedures, timing, and institutional details can support an assumption, but they do not turn its unobserved counterfactual claims into directly observed facts.
Common Mistake
Wrong reasoning: “The instrument groups have similar measured covariates and treatment uptake differs, so independence and exclusion are confirmed.” Similarity on measured covariates does not rule out differences in unmeasured causes or dependence on potential outcomes. A first-stage contrast demonstrates an observed treatment difference, not the absence of a direct instrument effect on the outcome.
Correction: Report the first-stage estimate and covariate comparisons as observable evidence. Explain whether the design implies balance for the covariates examined. Treat independence, exclusion, and monotonicity as assumptions requiring design-based or substantive justification, while acknowledging that the observed data alone cannot generally verify them.
Check Your Understanding
Answer each question by distinguishing what the observed data show from what an assumption claims.
- A study has higher treatment uptake in the \(Z=1\) group. What does this say about the sample first stage, and what does it not establish about exclusion?
- Why does balance in a baseline covariate follow from an assignment assumption that includes \(X\), but not necessarily from independence of potential treatments and outcomes alone?
- Could an observed association between \(Z\) and \(Y\) occur when exclusion holds? Explain the role of treatment.
- Why can two populations with different proportions of defiers have the same treatment rates under \(Z=0\) and \(Z=1\)?
- What is the limitation of finding no statistically detectable imbalance across several measured covariates?