Why Consistency Matters for an Instrumental Variable
An instrumental-variable comparison uses differences in outcomes across instrument assignments and relates them to differences in treatment receipt. For that comparison to have a clear causal interpretation, the labels “assigned” and “treated” must correspond to sufficiently well-defined interventions. If one label covers materially different interventions, or if one participant’s outcome depends on other participants’ assignments or treatments, the usual individual-level potential outcomes may not describe the data.
This tutorial focuses on those foundations. It distinguishes consistency from exclusion: consistency connects observed data to potential outcomes under the interventions that actually occurred; exclusion is a separate restriction on whether the instrument can affect outcomes other than through treatment. It also explains why the usual LATE interpretation requires no interference between participants.
Potential Outcomes, Consistency, and Exclusion
Begin with a binary instrument label \(Z_i\) and a binary treatment indicator \(D_i\) for participant \(i\). To keep assignment versions explicit, let \(A_i\) denote the exact assignment intervention received, such as a particular message or offer package, and let \(Z_i=g(A_i)\) be its coarser recorded label. Let \(T_i\) denote the version of treatment received, such as a particular dose or delivery format. Write \(D_i(a)\) for treatment receipt under assignment intervention \(a\), and \(T_i(a)\) for the treatment version that would be received under \(a\). Write \(Y_i(d,t,a)\) for the outcome if treatment status were set to \(d\), its version to \(t\), and the assignment intervention to \(a\).
This distinction is important. The observed outcome is linked by consistency to the joint potential outcome under both the received treatment and the received assignment. Only after imposing exclusion can assignment be removed from the outcome potential-outcome notation. If treatment versions have different effects, even exclusion does not by itself make the binary label \(D_i\) sufficient: the treatment version \(T_i\) still matters.
In the standard binary-instrument LATE setup, a familiar identification argument additionally requires instrument independence (possibly conditional on covariates), relevance, exclusion, and monotonicity, along with well-defined interventions and no interference. Under these conditions, the Wald ratio identifies an average treatment effect for compliers. The role of consistency and no interference here is to make the potential outcomes and comparisons in that argument meaningful, not to replace the other assumptions.
Worked Example: One Treatment Label, Two Treatment Versions
Suppose a hypothetical program records treatment as \(D_i=1\) whenever a participant receives tutoring. In fact, some treated participants receive a 30-minute group session and others receive a 60-minute individual session. For a particular hypothetical participant, suppose the potential outcomes under no tutoring, group tutoring, and individual tutoring are respectively 70, 74, and 82 points on a later assessment.
If the treatment is recorded only as the binary label \(D_i\), the notation \(Y_i(1)\) appears to refer to one outcome under “tutoring.” But the two versions give different outcomes: 74 points under group tutoring and 82 under individual tutoring. There is no single value of \(Y_i(1)\) for this participant unless the treatment intervention is further specified or one justifies that its versions are equivalent for the outcome.
For example, if this participant receives group tutoring, consistency gives an observed outcome of 74 points under the actual group intervention. It does not imply that the outcome equals 82 points, nor does it justify using a potential outcome for an unspecified version. The issue is not a calculation error; it is that the binary treatment label has not defined one intervention.
Versions of the Instrument
A coarse instrument label can conceal different assignment interventions too. Suppose \(Z_i=1\) is coded for anyone encouraged to attend a clinic, but some people receive an email and others a phone call. The exact intervention is \(A_i\), while the recorded instrument is only \(Z_i=g(A_i)\). If the versions differ in how they affect treatment uptake, or if one version affects the outcome directly, treating all of them as the same instrument may undermine the intended comparison.
For example, an email might simply provide information, whereas a phone call might also schedule an appointment. The appointment feature could increase treatment receipt, but it could also change an outcome through access to other services. In the latter case, exclusion fails for that version: even at the same treatment status and treatment version, the potential outcome may depend on whether the assignment was by email or phone. This is a distinct concern from inconsistency: the versions may be precisely documented, yet the exclusion restriction may still be false.
Worked Example: Two Versions of the Encouragement
Consider a hypothetical study in which the recorded instrument is \(Z=1\) for either an email or a phone encouragement, and \(Z=0\) for no encouragement. Suppose treatment status \(d=0\) means that the participant did not attend the clinic. For a particular participant, assume the potential outcome under no attendance is 10 points with no encouragement, 10 points after the email, and 12 points after the phone call.
The phone and email are both coded \(Z=1\), but the potential outcome differs by assignment version even when treatment is fixed at \(d=0\). In notation, \(Y_i(0,t,\text{phone})=12\) while \(Y_i(0,t,\text{email})=10\), for the relevant untreated status and treatment version. Thus the coarse \(Z=1\) label does not describe one outcome intervention, and exclusion across these assignment versions is not satisfied.
If the study instead defined separate instrument categories for email and phone, it could state and assess assumptions for each comparison. That change would not automatically establish independence, exclusion, or relevance; it would make the interventions being compared clearer. Merely pooling versions and calculating a Wald ratio cannot remove a direct effect or establish that versions are interchangeable.
The version problem can also be seen in a mean comparison. If the participants assigned \(Z=1\) receive different versions with proportions that vary across groups or settings, the mean outcome under the label \(Z=1\) is a mixture of version-specific outcomes. A change in that mixture can change the observed mean even if no single well-defined intervention changed. The instrument label then fails to isolate the causal contrast that the potential-outcome argument needs.
Interference Between Participants
Consistency alone does not rule out spillovers. Under interference, participant \(i\)’s outcome can depend on other participants’ treatments or assignments. Represent this with potential outcomes such as \(Y_i(\mathbf d,\mathbf t,\mathbf a)\), where the bold symbols denote the treatment statuses, treatment versions, and assignment interventions for the full group. The realized outcome is linked to the realized vectors by consistency, but it need not be a function only of participant \(i\)’s own treatment and assignment.
The usual LATE comparison treats each participant’s potential treatment and outcome as functions of that participant’s instrument value. If one participant’s instrument changes a peer’s treatment and thereby changes the first participant’s outcome, then the outcome difference by own instrument assignment includes a spillover pathway. It is no longer just the effect of own treatment among compliers. Random assignment of the instrument does not, by itself, eliminate this problem: randomized assignments can still generate causal spillovers.
Worked Example: A Spillover Changes the Wald Ratio
Consider a hypothetical setting with linked pairs. Let \(Z_i\) be participant \(i\)’s instrument and \(D_i\) their treatment, while \(D_j\) is the linked peer’s treatment. Suppose participant \(i\)’s outcome, measured in score points, is $$ Y_i=2D_i+3D_j. $$
The coefficient 2 means that changing only \(i\)’s treatment from 0 to 1 changes \(i\)’s outcome by 2 points. The coefficient 3 means that changing only the peer’s treatment from 0 to 1 changes \(i\)’s outcome by 3 points. Suppose the hypothetical assignment-group averages are as follows:
| Own instrument \(Z_i\) | Mean own treatment \(D_i\) | Mean peer treatment \(D_j\) | Mean outcome \(Y_i\) |
|---|---|---|---|
| 0 | 0.20 | 0.10 | 0.70 |
| 1 | 0.70 | 0.30 | 2.30 |
The outcome means agree with the stated outcome rule: for \(Z_i=0\), \(2(0.20)+3(0.10)=0.70\); for \(Z_i=1\), \(2(0.70)+3(0.30)=2.30\). The first stage is \(0.70-0.20=0.50\), and the reduced-form difference is \(2.30-0.70=1.60\). Therefore the Wald ratio is $$ \frac{E[Y_i\mid Z_i=1]-E[Y_i\mid Z_i=0]} {E[D_i\mid Z_i=1]-E[D_i\mid Z_i=0]} = \frac{1.60}{0.50}=3.20 \text{ points}. $$
This 3.20-point ratio is not the 2-point effect of changing only the participant’s own treatment. The assignment-group difference in mean outcome is \(2(0.50)+3(0.20)=1.60\): it includes a 0.60-point spillover contribution from the 0.20 increase in peer treatment. Dividing by the own-treatment first stage of 0.50 yields 3.20. This calculation shows exactly why the standard own-treatment LATE interpretation is not justified by these comparisons when interference operates.
How the Standard Comparison Depends on These Conditions
Under no interference and well-defined interventions, consistency allows observed group means to be interpreted as means of the potential outcomes under the assignments received. For a binary instrument, the observed outcome difference compares assignment-specific outcome means, while the first stage compares assignment-specific treatment means. Exclusion is then needed to attribute outcome changes to treatment rather than a direct instrument effect; independence and monotonicity support the usual complier interpretation.
A short justification shows why consistency matters at the first step. If exact assignment is \(A_i=a\), consistency gives \(Y_i=Y_i(D_i(a),T_i(a),a)\) for those actually assigned \(a\). Thus, under assignment to \(a\), the observed outcome mean corresponds to outcomes under that intervention, subject to the design’s assignment conditions. If instead the recorded label \(Z_i=1\) combines versions \(a_1\) and \(a_2\), its mean is a weighted mixture: $$ E[Y_i\mid Z_i=1] = P(A_i=a_1\mid Z_i=1)E[Y_i\mid A_i=a_1] + P(A_i=a_2\mid Z_i=1)E[Y_i\mid A_i=a_2]. $$
The mixture can be treated as one intervention only if the versions are irrelevant for the relevant potential outcomes or the estimand explicitly defines the mixture and its version proportions. Similarly, if \(Y_i\) depends on peers’ treatment, conditioning on own \(Z_i\) does not isolate changes in own treatment. These are failures in the causal comparison itself, not problems that can be fixed simply by a larger sample or a more precise standard error.
Diagnostics, Alternatives, and Limitations
Diagnostics begin with the intervention description, not a significance test. Write down what an assigned participant can actually receive and what counts as treatment. Check protocols, delivery records, timing, dose, provider, and access to ancillary services. Ask whether two participants coded \(D=1\) could receive materially different treatments, and whether two participants coded \(Z=1\) could receive different encouragements. Then ask whether peers can affect outcomes through shared resources, information, behavior, or treatment access.
When versions are consequential, possible responses include defining treatment or instrument categories more precisely, reporting version-specific comparisons, or explicitly defining a stochastic intervention that specifies the mixture of versions. When interference is plausible, the design and analysis may need to operate at the cluster level, or specify peer exposure and estimate effects under an explicit exposure mapping. These approaches require their own assumptions; relabeling variables alone does not solve the underlying problem.
The main alternatives answer different questions. A conventional individual-level IV analysis relies on no interference and sufficiently well-defined interventions to interpret its ratio as a complier effect. Version-specific analyses may clarify which intervention is being studied, but can be imprecise or confounded if versions were not assigned comparably. Cluster-level assignment can accommodate some spillover structures, but changes the unit and often the estimand. In all cases, observational records alone cannot prove that treatment versions are equivalent, that exclusion holds, or that interference is absent.
Common Mistakes
- Calling \(Y_i=Y_i(D_i)\) consistency without qualification. The general consistency link uses the joint potential outcome \(Y_i(D_i,T_i,A_i)\). Removing assignment requires exclusion, and removing treatment version requires that versions not matter for outcomes.
- Assuming a common label guarantees a common intervention. A binary “treated” or “encouraged” code can conceal different versions. Define the intervention precisely or explain why versions are outcome-equivalent.
- Treating randomization as a cure for spillovers. Randomization can support independence of assignment, but it does not prevent one participant’s assignment or treatment from affecting another participant’s outcome.
- Interpreting every Wald ratio as a LATE. A nonzero first stage is not enough. Without the relevant assumptions, including well-defined potential outcomes and no interference, the ratio need not be a complier effect.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why does \(Y_i=Y_i(D_i)\) generally require more than consistency in an IV design?
- A study codes both a text reminder and a transportation voucher as \(Z=1\). Name two distinct questions researchers should ask before treating them as one instrument intervention.
- If participant \(i\)’s outcome depends on a peer’s treatment, which no-interference condition fails?
- In the spillover example, calculate the contribution of the change in peer treatment to the reduced-form outcome difference.
- Does a statistically precise Wald ratio establish that consistency and no interference hold? Explain.
Solutions. (1) General consistency links the outcome to the joint potential outcome under received treatment, treatment version, and assignment, \(Y_i=Y_i(D_i,T_i,A_i)\). Exclusion is needed to remove assignment, and version irrelevance is needed to remove \(T_i\). (2) Ask whether the two versions affect treatment uptake differently and whether either can affect the outcome other than through the defined treatment. (3) No interference fails because changing the peer’s treatment while holding \(i\)’s own interventions fixed can change \(i\)’s outcome. (4) The peer treatment mean increases by \(0.30-0.10=0.20\), so its contribution is \(3(0.20)=0.60\) score points. (5) No. Precision describes sampling uncertainty under the analysis; it does not establish that intervention labels are well-defined or that participants cannot affect one another.