Plausibly Exogenous Instruments and Conley Sensitivity Analysis

When Exclusion Is Plausible but Not Certain

Instrumental-variable analyses often rely on an exclusion restriction: the instrument affects the outcome only by changing treatment. In an encouragement study, for example, assignment to receive an offer may change treatment uptake, and the exclusion restriction says that assignment has no other route to the outcome. The design may make this assumption credible without making it certain.

The plausibly exogenous approach makes that uncertainty explicit. Rather than setting the instrument’s direct effect on the outcome to exactly zero, specify a defensible range for it and calculate the treatment effects compatible with each value in that range. The result is a sensitivity analysis: it shows how the treatment-effect estimate depends on the strength and direction of a possible exclusion violation.

The Model and the Sensitivity Relationship

Let \(Z\) denote a binary randomized encouragement, \(D\) a binary treatment, and \(Y\) a quantitative outcome. Define the first-stage effect \(\pi\) as the difference in mean treatment uptake between the encouraged and unencouraged groups. Define the reduced-form effect \(\rho\) as the corresponding difference in mean outcome. Both contrasts are oriented as \(Z=1\) minus \(Z=0\).

Use an additive structural mean model in which the average outcome contrast can be decomposed into a treatment component and a direct instrument component:

Here \(\beta\) is the treatment effect on the outcome scale, and \(\gamma\) is the direct effect of encouragement on the outcome scale, apart from any effect operating through treatment. With a binary encouragement, \(\gamma\) is the direct outcome contrast for assignment to encouragement rather than no encouragement, holding the treatment-mediated contribution apart. If \(Y\) is measured in points, then \(\rho\) and \(\gamma\) are in points; if \(D\) is a treatment indicator, \(\pi\) is a proportion and \(\beta\) is in outcome points per treatment.

This decomposition requires more than arithmetic. The encouragement must be independent of the relevant potential outcomes and treatment responses, as would be supported by valid random assignment. The first stage must be nonzero (\(\pi\ne 0\)). The additive model must be appropriate for the target effect: for example, it is satisfied by a constant treatment effect and a common additive direct effect. In a binary-instrument setting with heterogeneous treatment effects, the corresponding interpretation can be a complier-average treatment effect if monotonicity holds and the direct effect is common across compliance types. If direct effects vary across people or compliance types, a single \(\gamma\) may not capture the needed decomposition; the analyst must justify the chosen model rather than assume the formula automatically applies.

Under the exact exclusion restriction, \(\gamma=0\), and the equation gives the usual Wald ratio \(\beta=\rho/\pi\). Under plausible exogeneity, \(\gamma\) is not fixed at zero. Rearranging the model for any specified value of \(\gamma\) gives:

Thus the sensitivity calculation is direct: subtract the posited direct effect from the reduced form, then divide by the first stage. The range of \(\beta(\gamma)\) as \(\gamma\) varies over a chosen interval is the implied treatment-effect sensitivity range.

Deriving the Effect Range

Suppose the analyst judges that the direct effect lies in \([\gamma_L,\gamma_U]\), where \(\gamma_L\le\gamma_U\). Because \(\beta(\gamma)\) is linear in \(\gamma\), its extrema over this interval occur at the endpoints. Its slope is \(-1/\pi\), so the endpoint order depends on the sign of the first stage.

If \(\pi>0\), this slope is negative: increasing the assumed direct effect reduces the treatment effect needed to explain the same reduced form. The implied range is therefore \([\beta(\gamma_U),\beta(\gamma_L)]\). If \(\pi<0\), the slope is positive, so the range is \([\beta(\gamma_L),\beta(\gamma_U)]\). Equivalently, calculate both endpoint values and report the smaller first.

Choosing a Direct-Effect Range and Finding a Tipping Point

The calculations are only as credible as the range supplied for \(\gamma\). The range should be expressed in the outcome’s units and justified using substantive knowledge about how assignment could affect the outcome apart from treatment. Relevant considerations include the content of the encouragement, possible changes in attention or access, and evidence about the size of comparable pathways. A symmetric interval around zero is convenient, but it is not required: there may be a reason to expect a direct effect to be mostly positive, mostly negative, or asymmetric.

A useful complementary question is: how large would the direct effect have to be for the treatment effect to reach a particular threshold? Solving \(\beta=(\rho-\gamma)/\pi\) for \(\gamma\) gives:

This value is a tipping point for the chosen treatment-effect threshold. It does not tell us whether that direct effect is plausible; that judgment comes from the substantive context.

Interpreting the Sensitivity Analysis

A concise report should give the observed reduced form and first stage, the assumed direct-effect range and its units, and the resulting treatment-effect range. It should also state the assumptions that connect these quantities. For example: “With a 0.30 increase in treatment uptake and a 4.8-point reduced-form increase, allowing a direct encouragement effect from \(-0.6\) to \(0.9\) points implies treatment effects from 13.0 to 18.0 points under the additive model.” This keeps the conclusion tied to both the data and the sensitivity choice.

The range calculated from point estimates does not account for sampling variability. A confidence interval for a treatment effect must also account for uncertainty in the reduced form and first stage; dividing uncertain estimates can be especially problematic when the first stage is weak. Therefore, do not label the point-estimate sensitivity range a confidence interval or infer that it has a particular coverage probability. If inferential intervals are reported, their construction must reflect uncertainty in the estimates and the sensitivity parameter, using methods appropriate to the strength of the instrument.

The method matters because exact exclusion can be scientifically demanding, while silently ignoring a possible direct pathway can make an instrumental-variable estimate look more certain than it is. Making \(\gamma\) explicit reveals the direction and scale of the assumption: for a positive first stage, a more positive direct effect leaves less of the reduced-form association to attribute to treatment; a negative direct effect leaves more. Reporting how the conclusion moves across a justified range is more transparent than presenting only the exact-exclusion estimate.

Common Mistakes

• Calling the sensitivity range a confidence interval. The endpoint calculations above vary \(\gamma\) while holding the estimated \(\rho\) and \(\pi\) fixed. They do not, by themselves, quantify sampling uncertainty.
• Reversing the endpoints without checking the first stage. With \(\pi>0\), increasing \(\gamma\) decreases \(\beta\); with \(\pi<0\), it increases \(\beta\). Calculate both values and order them numerically.
• Using incompatible units. The direct effect \(\gamma\) must be measured on the same outcome scale as \(\rho\). Dividing their difference by the treatment uptake contrast gives an outcome effect per treatment, not another outcome contrast per encouragement.
• Treating a chosen range as evidence. A range is an assumption informed by the design and substantive knowledge, not something guaranteed by the observed reduced form. Explain why its endpoints are plausible.
• Assuming the formula solves any form of exclusion violation. The decomposition requires a suitable additive model and a defensible interpretation of \(\gamma\). Heterogeneous or type-specific direct effects may not be represented by one common value.