Auditing Assumptions in an Encouragement Design

Why an Encouragement Design Needs an Assumption Audit

An encouragement design randomly varies an offer, referral, reminder, or other prompt intended to change whether people receive a treatment. Randomizing the encouragement can make treatment receipt more informative for causal analysis, but it does not automatically make treatment receipt randomized. The design supports an instrumental-variables interpretation only if the assignment mechanism, treatment response, and outcome pathways satisfy additional assumptions.

Consider a hypothetical employment program. Eligible job seekers are randomly assigned either an offer of a priority appointment for job-search counseling or the usual information about the program. Some people offered an appointment do not attend; some not offered one find another way to attend. The encouragement is the instrument, attendance is treatment, and employment six months later is the outcome. Auditing the design means asking not just whether the offer changed attendance, but also how assignment occurred, whether the offer could affect employment without attendance, and whether it could discourage anyone who would otherwise attend.

Four Questions for the Audit

The assumptions concern different links in the study’s causal structure. Independence concerns how \(Z\) is assigned. Relevance concerns whether \(Z\) changes \(D\). Exclusion concerns whether \(Z\) can change \(Y\) through a route other than \(D\). Monotonicity concerns the direction of each individual’s treatment response to the encouragement. It is useful to keep these questions separate: a randomized assignment can be relevant but fail exclusion, and a strong average first stage does not establish monotonicity.

Random assignment is a design-based reason to regard independence as plausible; it is not a claim that every realized sample has exactly identical baseline characteristics. Conversely, similar observed characteristics do not establish independence: potential outcomes and potential treatment responses are not all observed, and unmeasured differences can remain. The audit should examine the randomization protocol, allocation records, concealment where relevant, exclusions, attrition, and whether analysis conditions on variables affected by assignment.

For the employment example, the priority appointment might increase counseling attendance, which could change employment. That is the intended path, \(Z\rightarrow D\rightarrow Y\). But if the offer also supplies job listings, signals that an agency considers someone especially employable, changes access to staff, or motivates job searching even among people who never attend, it may affect employment outside the defined treatment. The treatment must be specified precisely: if “treatment” means attendance at counseling, information supplied directly by the offer is not automatically part of that treatment.

A positive first stage is expected when the encouragement is designed to increase take-up, although relevance as a mathematical condition requires a nonzero difference. The estimated first stage is evidence about the study population, not proof that every individual responds in the same way. Its size and precision matter for the credibility and stability of estimation; the conceptual relevance condition itself is simply nonzero assignment-induced variation in treatment.

Monotonicity is an individual-level claim, not a statement that average take-up rises. A positive average first stage can coexist with some people being discouraged, provided more people are induced to take treatment than are deterred. The substantive audit should therefore ask whether the encouragement could crowd out other access, create stigma, impose a burden, or otherwise discourage treatment among people who would have received it without the offer.

A Short Justification for Why the Assumptions Work Together

Under independence, the assignment groups identify the corresponding averages of potential quantities. Under exclusion, observed outcomes can be written as \(Y_i(D_i(Z_i))\). The reduced-form assignment contrast is therefore the average outcome change induced by assignment. Monotonicity tells us which people can contribute to that change.

With monotonicity, \(D_i(1)-D_i(0)\) equals one for compliers and zero for everyone else. Thus the reduced form is the complier share multiplied by the average treatment effect among compliers. The first stage is the complier share, because

If this first stage is nonzero, dividing the reduced form by it yields the average treatment effect among compliers. This justification shows why an audit cannot focus on the first stage alone: the interpretation also relies on independence, exclusion, and monotonicity.

A Practical Audit Sequence

An effective audit records the evidence and the unresolved assumptions separately. Use the following sequence to organize the review.

What the Data Can—and Cannot—Establish

Assignment records can document a randomization procedure, and observed take-up can show whether the encouragement changed treatment receipt in the sample. Outcome and process data may reveal a plausible exclusion violation, such as a service delivered directly to nonparticipants. But a pathway that is not measured may remain undetected. Likewise, monotonicity concerns pairs of potential treatment statuses, only one of which is observed for each person; it is generally assessed through design knowledge and behavioral reasoning rather than verified from the observed take-up difference.

Avoid treating a non-significant baseline comparison as proof of independence, a large first stage as proof of validity, or a positive average take-up difference as proof that no defiers exist. Each statement confuses evidence about one feature with a different assumption. A careful report identifies the specific design fact or empirical pattern supporting each assumption, names plausible threats, and explains what remains uncertain.