Why Exclusion Needs Special Attention in Encouragement Designs
An encouragement design assigns an offer, reminder, invitation, or incentive intended to change whether people receive a treatment. Random assignment can make the offer independent of potential outcomes, but that fact alone does not establish that the offer affects outcomes only through treatment receipt. An invitation may also change expectations, provide useful information, or give people resources that affect the outcome directly.
The key assessment is therefore about the causal pathways created by the actual offer. Before interpreting an instrumental-variable estimate as an effect of treatment receipt, ask what a person assigned the offer experiences besides any resulting change in receipt. A credible answer requires detailed knowledge of how the intervention is delivered, not just evidence that assignment was randomized.
The definition holds treatment receipt fixed. It does not say the encouragement must have no effect on the outcome overall: the encouragement may change \(D\), which may change \(Y\). Rather, exclusion rules out an additional route from \(Z\) to \(Y\) that remains when \(D\) is held fixed.
Separate Exclusion From Randomization
Randomization concerns how \(Z\) is assigned. In a randomized design, assignment is independent of the relevant potential outcomes and potential treatment receipt. Exclusion concerns what assignment can do after it occurs. These are distinct claims: assignment can be perfectly randomized while the offer itself has a direct effect on the outcome.
For example, a randomized invitation to a job-training program might include a transportation voucher. The voucher could increase participants’ ability to attend job interviews, even if they never attend training. Randomization can make voucher assignment unrelated to participants’ pre-existing employability, but it cannot make the voucher’s own effect disappear. If the treatment of interest is training attendance, this is a possible exclusion violation.
| Design feature or finding | What it bears on | What it does not establish |
|---|---|---|
| Random allocation of the offer | Independence of \(Z\) from potential outcomes and treatment receipt | That the offer affects outcomes only through \(D\) |
| A first-stage difference in treatment receipt | Relevance: the offer shifts \(D\) | That the offer has no direct outcome effect |
| A plausible offer-related pathway to \(Y\) without changing \(D\) | A substantive threat to exclusion | That the size of the direct effect is known |
| No detected difference on a selected placebo outcome | Some evidence about that particular outcome and measurement | That exclusion holds for the study’s outcome |
Worked Example: A Randomized Offer With a Direct Route
Suppose, hypothetically, that people are randomly assigned an offer of a financial-literacy course. Those assigned the offer receive a \$40 budgeting credit whether or not they attend. Let \(D=1\) mean course attendance, and let \(Y\) be a financial-wellbeing score in points. Assume the offer increases course attendance by 0.20, the average outcome contrast between assignment groups is 1.50 points, and the credit directly raises the score by 0.60 points at fixed attendance. For this illustration, assume random assignment, monotonicity, and a constant direct effect of 0.60 points.
The offer’s direct effect is present even for a person whose attendance is held fixed: the credit can improve financial wellbeing without course attendance. Thus \(Y(1,d)-Y(0,d)=0.60\) points, which contradicts exclusion.
Under the stated assumptions, the reduced-form contrast combines the treatment-mediated effect and this direct effect. If \(\tau_C\) is the average course effect for compliers, the contrast is \(0.20\tau_C+0.60\). Substituting the hypothetical reduced form gives:
The ordinary Wald ratio would instead be \(1.50/0.20=7.50\) points. That ratio is not the course effect under these assumptions: it includes the credit’s direct effect, magnified by division by the 0.20 first stage. The adjusted value of 4.50 points follows only because the example stipulates that the direct effect is known and constant; real data do not generally supply that knowledge.
Trace the Offer’s Possible Pathways
A practical assessment begins by drawing or describing the routes from assignment to outcome. The intended route is \(Z\rightarrow D\rightarrow Y\). Exclusion is threatened by another route, such as \(Z\rightarrow Y\), or by a route through a variable changed by the offer that affects \(Y\) without being part of the defined treatment. Common possibilities include:
- Material benefits: money, vouchers, equipment, transport, or priority access provided with the offer.
- Information and expectations: a reminder or invitation may teach participants something, change beliefs, or alter motivation.
- Contact and attention: additional calls or staff contact may provide support, reassurance, or monitoring.
- Changes to other services: an offer may lead participants or providers to seek, delay, or alter care outside the treatment being studied.
- Stigma or burden: being contacted, labelled, or asked to respond may affect wellbeing or behavior regardless of treatment receipt.
These are not automatic violations. The relevant question is whether the route could change the measured outcome while holding the defined treatment fixed. For instance, a reminder that only prompts a person to attend the specified treatment may operate through \(D\). The same reminder may violate exclusion if it also supplies actionable advice that changes the outcome among people who do not attend.
Worked Example: A Reminder That Also Gives Advice
Imagine a hypothetical trial in which adults are randomly assigned a text-message invitation to a nutrition program. Treatment receipt \(D\) is defined as attending at least one program session, and \(Y\) is weekly fruit and vegetable intake. The text both invites people to attend and gives a practical meal-planning tip. Consider a person who does not attend, so \(D=0\) under either assignment.
If the tip changes that person’s intake, then the fixed-treatment contrast is not zero: \(Y(1,0)-Y(0,0)\ne 0\). The tip provides a route from assignment to the outcome that does not pass through the defined treatment receipt. Randomization does not remove this route; it ensures that offer assignment is not selected according to people’s underlying potential intake.
A possible redesign is to send both groups the same meal-planning information and vary only the invitation to attend. Alternatively, define the intervention as the package of invitation plus advice, but then treatment receipt must be defined so the instrument is not separately changing a component of the outcome-relevant package among people classified as untreated. A broader label alone does not repair the causal design: the definition of \(D\) must match what participants actually receive, and the resulting treatment contrast must be meaningful.
A Short Derivation: How a Direct Effect Enters the Wald Ratio
The following calculation makes the consequence of one form of exclusion failure explicit. Assume \(Z\) is randomized, \(D(z)\) is potential treatment receipt under assignment \(z\), monotonicity holds, and the direct effect is constant and additive: \(Y(z,d)=Y(d)+\gamma z\). Let \(C\) denote the complier group, the people with \(D(1)=1\) and \(D(0)=0\). Under monotonicity, the increase in the treatment rate is the complier share, \(P(C)\).
Randomization makes the average reduced-form contrast equal to the average contrast in the assignment-specific potential outcomes. The treatment-mediated component is zero for always-takers and never-takers, whose treatment receipt does not change with \(Z\); among compliers, it is the complier share times their average treatment effect, \(\tau_C\). The constant direct effect contributes \(\gamma\) for everyone. Therefore:
The first stage is \(E[D\mid Z=1]-E[D\mid Z=0]=P(C)\). Dividing the reduced form by the first stage yields:
Thus even a modest direct effect can materially alter the ratio when the first stage is small. If \(\gamma=0\), this expression reduces to the usual complier-effect interpretation under the other stated assumptions. If the direct effect varies across people, this simple formula need not apply; the reduced form still contains any assignment effect not operating through treatment, but its decomposition requires additional assumptions.
Worked Example: A Small First Stage Magnifies a Direct Effect
Suppose, hypothetically, an invitation to a preventive-care appointment increases appointment attendance by 0.10. The invitation also causes an average 0.20-point increase in a health-information score at fixed attendance. Assume the remaining IV assumptions and the constant additive direct effect model above hold, and suppose the complier treatment effect is 3.00 points.
The reduced-form contrast is:
The Wald ratio is \(0.50/0.10=5.00\) points, while the complier treatment effect stipulated in the example is 3.00 points. The difference is \(0.20/0.10=2.00\) points. This calculation is not a general correction procedure: in practice, the direct effect is usually unknown. It shows why a weak first stage can make even a small direct pathway consequential.
Diagnostics, Design Choices, and Limits
Exclusion is a substantive assumption about counterfactual outcomes, not a condition that can generally be confirmed from the observed distribution of \(Z\), \(D\), and \(Y\) alone. A useful assessment combines design documentation, subject-matter knowledge, and targeted empirical checks. Ask what each group actually receives, whether contact differs, whether control participants can access the same information or resources, and whether the outcome could respond to any of those differences independently of treatment receipt.
Some checks can reveal problems. For example, an offer effect on an outcome measured before treatment could indicate a direct pathway, a measurement or reporting effect, or a design or data problem. An offer effect on a carefully chosen outcome that treatment could not plausibly affect may also raise concern. But a null result on a placebo outcome does not prove exclusion for the primary outcome: the alternative pathway may affect one outcome and not the other, or the check may lack precision. Nor can simply comparing outcomes among observed nonrecipients generally isolate a direct effect; receipt after assignment is not a randomized subgroup characteristic.
When a plausible alternative pathway appears, possible responses include removing the extra component, giving it equally to both assignment groups, measuring and reporting it, or redefining treatment receipt to include the relevant package when that definition is substantively coherent. Sensitivity analysis can show how conclusions would change under assumed direct effects, but it cannot identify the true direct effect without additional information or assumptions.
Compared with an encouragement design, directly randomizing treatment receipt avoids the specific problem of using an offer as an instrument, when such randomization is feasible and ethical. A randomized offer remains valuable when uptake cannot be assigned, but its IV interpretation needs exclusion as well as randomization and a change in treatment receipt. Ordinary randomized comparisons estimate the effect of assignment to the offer, not automatically the effect of treatment receipt. The distinction matters: a design can validly estimate an offer effect while failing to support the intended IV estimate.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A randomized text invitation increases program attendance and provides a \$10 phone credit. Why does randomization not by itself establish exclusion?
- In the constant direct-effect model, the first stage is 0.25, the reduced-form contrast is 1.40 outcome units, and \(\gamma=0.15\) outcome units. Calculate the implied \(\tau_C\).
- Why does finding no offer effect on one placebo outcome fail to prove exclusion for the primary outcome?
- Suppose a reminder changes \(Y\) only by increasing receipt of the specified treatment. Is that pathway itself a violation of exclusion? Explain.
Solutions. (1) The credit may affect the outcome even for people whose program attendance is fixed. Randomization concerns the assignment process, not whether the assigned offer has a direct effect. (2) From the stated model, \(1.40=0.25\tau_C+0.15\), so \(\tau_C=(1.40-0.15)/0.25=5.00\) outcome units. (3) The alternative pathway may affect the primary outcome but not the selected placebo, and a null finding may also be imprecise. (4) No. If the reminder affects the outcome only through the defined treatment receipt, that is the intended mediated pathway; exclusion rules out a separate effect at fixed receipt.