Sensitivity Curves For Treatment Effect Bounds

Why Plot Treatment-Effect Bounds?

A sensitivity analysis can show more than whether a causal conclusion changes at one chosen bias limit. Plotting both treatment-effect endpoints over a range of limits reveals how quickly the identified interval expands, whether its location is mainly positive or negative, and when the data and assumptions no longer determine the effect’s sign. The curve makes the role of the selection-bias assumption visible rather than hiding it in a single reported interval.

This tutorial uses the average treatment effect on the treated as its target. The same plotting logic applies whenever the effect bound is expressed through an estimand and a bounded bias term, but the formulas must be derived for that estimand rather than borrowed automatically.

Assume the treatment and outcome are well-defined, the conditional means in these definitions exist, and consistency holds: a treated individual’s observed outcome is \(Y(1)\), while an untreated individual’s observed outcome is \(Y(0)\). No assumption of random treatment assignment is made. In particular, \(B\) may be nonzero: the treated and untreated groups can differ in their mean outcome even if neither group received treatment.

From a Bias Limit to Two Curves

By consistency, the treated group’s observed mean is \(E[Y(1)\mid D=1]\), and the untreated group’s observed mean is \(E[Y(0)\mid D=0]\). Adding and subtracting \(E[Y(0)\mid D=1]\) gives the key relationship:

Thus \(\tau_{\mathrm{ATT}}=\delta-B\). A selection-bias limit does not specify the value of \(B\); it restricts the values considered plausible. Under the symmetric limit \(|B|\leq\Lambda\), where \(\Lambda\geq0\) is measured in the same units as the outcome, the smallest possible effect occurs at the largest allowed bias, \(B=\Lambda\). The largest possible effect occurs at \(B=-\Lambda\).

These conclusions follow directly from the endpoints of the allowed bias range. Because subtracting \(B\) reverses order, the lower endpoint is \(\delta-\Lambda\), not \(\delta+\Lambda\). Both endpoints are straight lines in \(\Lambda\): the lower one has slope \(-1\), and the upper one has slope \(+1\). Their distance grows at two outcome units for every one-unit increase in the symmetric bias limit. Their shared midpoint stays fixed because the symmetric restriction expands equally in both directions.

Reading the Location as Well as the Width

Width describes how much uncertainty the sensitivity restriction permits; location describes which effect values remain possible. Under a symmetric limit, location is especially easy to read because the midpoint remains \(\delta\). If \(\delta\) is negative, the interval begins on the negative side and expands toward zero and beyond as \(\Lambda\) increases. If \(\delta\) is positive, the reverse sign interpretation applies. At \(\Lambda=0\), the interval collapses to \(\delta\), which corresponds to imposing \(B=0\), not to establishing that treatment assignment was random.

A common generalization allows an asymmetric bias range \(B_L(\lambda)\leq B\leq B_U(\lambda)\), where \(\lambda\) indexes the sensitivity restriction. The effect interval is then \([\delta-B_U(\lambda),\ \delta-B_L(\lambda)]\). Its midpoint is \(\delta-\{B_L(\lambda)+B_U(\lambda)\}/2\), and its width is \(B_U(\lambda)-B_L(\lambda)\). If the permitted bias range shifts as well as expands, the effect interval’s midpoint can move. In that setting, the endpoints need not have slopes of exactly \(+1\) and \(-1\).

Outcome Support Can Make Curves Flatten

The straight-line result assumes the stated bias range is the only restriction relevant to the effect. If the outcome has known bounds, such as \(a\leq Y\leq b\), the counterfactual mean \(E[Y(0)\mid D=1]\) must also lie in \([a,b]\). Since \(E[Y(0)\mid D=1]=E[Y\mid D=0]+B\), feasible bias values must satisfy \(a-E[Y\mid D=0]\leq B\leq b-E[Y\mid D=0]\). The allowed bias values are the intersection of this feasible range and the sensitivity range. The corresponding effect interval can therefore be narrower than the unrestricted straight-line interval, and its endpoints can stop moving once a support constraint binds.

Constructing and Checking a Sensitivity Plot

For a symmetric restriction, the horizontal axis is \(\Lambda\), in outcome units, and the vertical axis is the treatment effect in the same units. Plot \(L(\Lambda)\) and \(U(\Lambda)\) over a substantively justified range, and shade the interval between them if useful. Label the zero-effect line when sign robustness matters. If outcome support or other restrictions apply, plot the endpoints after enforcing those restrictions rather than extending the unrestricted lines past feasible values.

A useful diagnostic is to check whether the plotted intervals are nested as the assumptions are relaxed. If larger \(\Lambda\) permits every bias value allowed at smaller \(\Lambda\), the identified sets cannot shrink. For the unrestricted symmetric case, check that the endpoints equal \(\delta\) at zero, the midpoint remains \(\delta\), and the width equals \(2\Lambda\). A curve that violates these checks may reflect a sign error, mismatched units, a changing bias center, or an additional constraint that has not been described.

The sensitivity limit should be calibrated in context. A value of 5 points has different substantive meaning for a score measured from 0 to 10 than for one measured from 0 to 100. The introductory material on choosing and calibrating a bias limit motivates that judgment; the plot displays its implications but cannot justify the chosen range by itself.

What the Curves Do—and Do Not—Show

A point estimate that sets \(B=0\) reports \(\delta\), the value at which the symmetric sensitivity interval collapses. A sensitivity curve instead shows the range of effects permitted as the bias restriction is relaxed. A tipping-point calculation identifies a particular limit at which an interval first includes a threshold such as zero; the curve provides the broader picture around that threshold. These are complementary summaries, not competing calculations.

The plotted interval is an identified set conditional on the assumptions and the population quantities used in the calculation. It is not a confidence interval and does not account for sampling uncertainty. In practice, observed group means are estimated from data, and uncertainty about estimated endpoints requires separate inferential methods. A wide interval does not mean that each value inside it is equally likely; it means that the stated restrictions do not rule those values out.

• Do not treat zero bias as proven. Setting \(\Lambda=0\) imposes \(B=0\); it is not evidence that the groups are comparable on untreated potential outcomes.
• Do not reverse the endpoints. Because \(\tau_{\mathrm{ATT}}=\delta-B\), the largest allowed bias produces the lower effect endpoint.
• Do not read a bound as a probability distribution. The interval describes compatible effect values, not how probable each value is.
• Do not extrapolate past feasibility. Known outcome support can constrain bias and flatten endpoints.
• Do not confuse robustness with precision. Whether the interval excludes zero concerns sign robustness under assumptions; its width alone is not a measure of sampling precision.