Sensitivity-and-Bounds Workshop for a Compromised Encouragement Study

Why Analyze a Compromised Encouragement Study?

An encouragement can change treatment uptake without assigning treatment itself. An instrumental-variable analysis can use that change to learn about treatment effects, but the usual interpretation depends on demanding conditions. In particular, the encouragement must not affect the outcome except through treatment, and it must not make some people less likely to take treatment while making others more likely. When either condition is doubtful, a single Wald estimate may give a misleading impression of certainty.

This workshop uses one hypothetical binary-outcome study to examine both problems together. We will allow a bounded average direct effect of the encouragement and a bounded share of defiers. The calculations give a sensitivity estimate under selected assumptions and an envelope of possible complier effects under ranges of assumptions. They do not remove the need to defend those assumptions.

Study Setup and Estimands

Let \(Z\) indicate assignment to an encouragement (\(Z=1\)) or no encouragement (\(Z=0\)), let \(D\) indicate treatment receipt, and let \(Y\) be a binary outcome. Write \(D(z)\) for the treatment that a person would receive under encouragement assignment \(z\), and \(Y(d,z)\) for the outcome under treatment level \(d\) and encouragement level \(z\). The observed values satisfy \(D=D(Z)\) and \(Y=Y(D(Z),Z)\).

Assume for the analysis that assignment is randomized, so \(Z\) is independent of the potential treatment and outcome values; that consistency holds; and that one person's assignment does not affect another person's outcome. These conditions allow the observed differences by assignment to represent population contrasts in potential outcomes. The central complication is that we do not assume exclusion: \(Y(d,1)\) may differ from \(Y(d,0)\), even when treatment is held fixed.

Definition: A complier has \(D(1)=1\) and \(D(0)=0\); a defier has \(D(1)=0\) and \(D(0)=1\). Always-takers have \(D(1)=D(0)=1\), and never-takers have \(D(1)=D(0)=0\). The complier treatment effect used here is \(\tau_C=E[Y(1,0)-Y(0,0)\mid D(1)=1,D(0)=0]\): the average treatment effect among compliers when the encouragement is held at \(0\).

The outcome is binary, so outcome differences and treatment effects are measured on the risk-difference scale. A value of \(0.10\), for example, means a 10-percentage-point difference in outcome probability. The complier effect at \(Z=0\) is a well-defined target in this setup, even though direct effects of encouragement are allowed.

Worked Example: Calculate the Observed Contrasts

Suppose a hypothetical randomized study assigns 500 people to each encouragement arm. In the \(Z=0\) arm, 150 receive treatment and 110 have the outcome. In the \(Z=1\) arm, 300 receive treatment and 170 have the outcome. These counts describe the hypothetical data; they are not evidence from a real study.

The treatment uptake rates are \(150/500=0.300\) without encouragement and \(300/500=0.600\) with encouragement. Thus the first-stage risk difference is

\Delta_D=P(D=1\mid Z=1)-P(D=1\mid Z=0)=0.600-0.300=0.300.

The outcome rates are \(110/500=0.220\) and \(170/500=0.340\), respectively. The outcome contrast by assignment is therefore

\Delta_Y=P(Y=1\mid Z=1)-P(Y=1\mid Z=0)=0.340-0.220=0.120.

The Wald ratio is \(0.120/0.300=0.400\), or 40.0 percentage points. It is a causal estimate of the complier effect only under further assumptions, notably exclusion and monotonicity. The positive first stage tells us that encouragement increases treatment uptake on average; by itself it does not establish those assumptions.

Separate the Direct Effect From the Treatment Pathway

To examine exclusion sensitivity, define the average direct effect contrast

\Gamma=E[Y(D(1),1)-Y(D(1),0)].

This definition compares outcomes under the two encouragement levels while holding treatment at the value a person would receive under \(Z=1\). It is a particular population-average direct-effect contrast, chosen so that the following decomposition is exact. A sensitivity range for \(\Gamma\) must refer to this contrast; it should not be silently substituted for a different direct-effect quantity.

For each person, add and subtract \(Y(D(1),0)\) in the outcome contrast between assignment levels. Random assignment and consistency then give the population identity

\Delta_Y = \Gamma + E[Y(D(1),0)-Y(D(0),0)].

The second term is the treatment-mediated contrast under \(Z=0\). Let \(\tau_0=Y(1,0)-Y(0,0)\). Because \(D\) is binary, \(Y(D(z),0)=Y(0,0)+D(z)\tau_0\), so the mediated contrast equals \(E[(D(1)-D(0))\tau_0]\). The difference \(D(1)-D(0)\) is 1 for compliers, -1 for defiers, and 0 for always-takers and never-takers. Consequently,

\Delta_Y-\Gamma=\pi_C\tau_C-\pi_F\tau_F,

where \(\pi_C\) and \(\pi_F\) are the population shares of compliers and defiers, and \(\tau_F=E[\tau_0\mid D(1)=0,D(0)=1]\) is the average treatment effect at \(Z=0\) among defiers. The first stage obeys \(\Delta_D=\pi_C-\pi_F\). This derivation shows why the Wald ratio need not equal the complier effect: a direct effect contributes to \(\Delta_Y\), and defiers contribute with the opposite sign from compliers.

Worked Example: Sensitivity Estimates With No Defiers

First consider the illustrative assumption that there are no defiers, so \(\pi_F=0\) and \(\pi_C=\Delta_D=0.300\). Suppose subject-matter reasoning supports examining direct-effect values from \(-0.030\) to \(0.030\). These values represent an average direct effect ranging from a 3-percentage-point reduction to a 3-percentage-point increase in the binary outcome risk.

Under no defiers, the decomposition reduces to \(\Delta_Y-\Gamma=\pi_C\tau_C\), so the sensitivity estimate is \((\Delta_Y-\Gamma)/\Delta_D\). At the lower direct-effect value, \(\Gamma=-0.030\),

\widehat{\tau}_C(-0.030) = \frac{0.120-(-0.030)}{0.300} = \frac{0.150}{0.300} = 0.500.

At \(\Gamma=0\), the estimate is \((0.120-0)/0.300=0.400\). At the upper value, \(\Gamma=0.030\),

\widehat{\tau}_C(0.030) = \frac{0.120-0.030}{0.300} = \frac{0.090}{0.300} = 0.300.

Thus, with no defiers and the stated direct-effect range, the sensitivity estimates run from 0.300 to 0.500: an estimated 30- to 50-percentage-point increase in outcome risk among compliers. This is a range of point estimates conditional on the assumptions, not a confidence interval and not a probability statement about where the true effect lies.

Allow a Small Share of Defiers

Now suppose monotonicity is uncertain, but the defier share is believed to be no greater than \(0.040\). Since \(\Delta_D=\pi_C-\pi_F=0.300\), setting \(f=\pi_F\) gives \(\pi_C=0.300+f\). The decomposition can therefore be rearranged as

\tau_C=\frac{\Delta_Y-\Gamma+f\tau_F}{0.300+f}.

Because the outcome is binary, the difference between two potential outcome values lies between \(-1\) and \(1\), so \(-1\leq\tau_F\leq1\). Take the sensitivity restrictions to be \(-0.030\leq\Gamma\leq0.030\) and \(0\leq f\leq0.040\). For any fixed \(f\), the smallest value of \(\tau_C\) occurs at \(\Gamma=0.030\) and \(\tau_F=-1\); the largest occurs at \(\Gamma=-0.030\) and \(\tau_F=1\). This gives the candidate endpoints

L(f)=\frac{0.090-f}{0.300+f}, \qquad U(f)=\frac{0.150+f}{0.300+f}.

The lower expression decreases as \(f\) increases: its derivative is \(-0.390/(0.300+f)^2\), which is negative. The upper expression increases as \(f\) increases: its derivative is \(0.150/(0.300+f)^2\), which is positive. The widest envelope therefore uses the largest allowed defier share, \(f=0.040\).

Worked Example: Combine the Direct-Effect and Defier Bounds

Substitute the worst-case allowed values into the lower endpoint. The most adverse combination for a positive complier effect is \(\Gamma=0.030\), \(f=0.040\), and \(\tau_F=-1\):

\tau_C \geq \frac{0.120-0.030+0.040(-1)}{0.300+0.040} = \frac{0.050}{0.340} = 0.1471.

For the upper endpoint, use \(\Gamma=-0.030\), \(f=0.040\), and \(\tau_F=1\):

\tau_C \leq \frac{0.120-(-0.030)+0.040(1)}{0.300+0.040} = \frac{0.190}{0.340} = 0.5588.

Under the stated restrictions, the resulting algebraic envelope is approximately \(0.147\) to \(0.559\), or a 14.7- to 55.9-percentage-point increase in outcome risk among compliers. The lower endpoint remains positive. In this hypothetical example, a positive complier effect is robust to the specified direct-effect and defier ranges, provided the randomization, consistency, and other setup assumptions hold. The breadth of the envelope also shows that the exact magnitude is sensitive to the assumptions.

What the Calculation Does and Does Not Establish

The result is a sensitivity bound based on the observed contrasts and the chosen restrictions. It is not automatically a sharp bound for every possible joint distribution of potential outcomes: additional compatibility restrictions from the full observed data could narrow the feasible set. Nor does the calculation account for sampling uncertainty. The proportions here are treated as inputs; in an actual study, uncertainty in the estimated contrasts would also need consideration before making a population-level claim.

The key scientific work is justifying the ranges. The data in this example do not reveal the average direct effect \(\Gamma\) or identify the defier share \(f\). They are sensitivity parameters. Researchers should explain why a range such as \(\pm 0.030\) is plausible on the outcome risk-difference scale and why a defier share of at most 0.040 is credible. If defensible ranges are wider, the resulting conclusion may be weaker.

Common Mistakes

Calling the Wald ratio the complier effect without qualification. The ratio \(0.400\) is the observed outcome contrast divided by the first stage. It equals the complier effect only under conditions such as exclusion and no defiers; the sensitivity analysis explicitly relaxes these conditions.
Subtracting the direct effect but ignoring defiers. The adjustment \((\Delta_Y-\Gamma)/\Delta_D\) is justified when there are no defiers. With defiers, the mediated contrast includes \(-\pi_F\tau_F\), so a bound or additional assumption is needed.
Treating a sensitivity range as a confidence interval. The interval-like envelope above describes effects compatible with specified sensitivity restrictions at the calculated contrasts. It does not by itself quantify repeated-sampling uncertainty.
Interpreting a positive estimate as a proven benefit for everyone. The target is the average effect among compliers, not the effect for all participants. A positive bound supports a positive complier effect only under the stated assumptions and ranges.

Key takeaway: With a binary outcome, a bounded direct-effect contrast and a bounded defier share can be carried explicitly through the IV decomposition. Here the resulting sensitivity envelope stays above zero, supporting a positive complier effect conditional on the assumptions, while leaving substantial uncertainty about its magnitude.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

For the hypothetical study, calculate the Wald ratio and state the additional assumptions needed to interpret it as the complier effect.
With no defiers and \(\Gamma=0.020\), what is the sensitivity estimate for \(\tau_C\)? Show the substitution.
Why does a defier's contribution to the treatment-mediated contrast have a minus sign?
If the upper bound on the defier share were increased, would the lower endpoint in this example rise or fall? Explain using \(L(f)\).
Does the calculated envelope account for sampling uncertainty? What does it describe instead?