Why Sensitivity Curves Matter
A partial-identification analysis often depends on assumptions that cannot be verified from the observed data alone. Sensitivity analysis makes this dependence explicit: rather than reporting one bound under one fixed assumption, we vary a parameter that measures how much a restriction is relaxed and show how the identified set changes. A sensitivity curve traces one or both endpoints of that set as the sensitivity parameter varies.
A tipping point is a value of the sensitivity parameter at which a stated conclusion changes. The conclusion might be that an effect is positive, that it exceeds a policy threshold, or that it lies within a substantively important range. A tipping point does not establish which sensitivity value is correct. It reports how strong a departure from the baseline assumptions would be needed to alter the conclusion.
The sensitivity parameter needs a clear scale and interpretation. For example, it may be a maximum confounding bias measured in percentage points, a maximum difference between respondent and nonrespondent means measured in outcome units, or a maximum direct effect of an instrument. A numerical tipping point is meaningful only alongside that interpretation and its units.
How Bound Curves Move
Many sensitivity analyses use nested restrictions: increasing \(\lambda\) allows a larger departure from the baseline model. If \(\lambda_1\leq\lambda_2\), and every model allowed at \(\lambda_1\) is also allowed at \(\lambda_2\), then \(\Theta(\lambda_1)\subseteq\Theta(\lambda_2)\). For interval-valued sets, this implies that the lower endpoint cannot rise and the upper endpoint cannot fall.
This conclusion follows directly from set inclusion. The smallest value in the larger set can be no larger than the smallest value in the smaller set, and the largest value in the larger set can be no smaller than the largest value in the smaller set. The endpoints may stay fixed over part of the range, or change at different rates. In particular, formulas obtained before imposing the target parameter’s natural range must not be extended beyond that range.
For a parameter with known support \([a,b]\), every valid identified set must lie inside \([a,b]\). Thus, if a sensitivity calculation first gives an interval \([L_0(\lambda),U_0(\lambda)]\), the reported set must account for the support restriction. When the only additional restriction is that the parameter lies in \([a,b]\), this amounts to intersecting the intervals:
provided the intersection is nonempty. This adjustment is essential: an interval endpoint outside the parameter’s possible range cannot be a sharp bound.
Worked Example: Confounding Bias and the Sign Tipping Point
Worked Example: Confounding Bias and the Sign Tipping Point
Suppose a hypothetical analysis reports an adjusted treatment-outcome contrast of \(d=4.8\) percentage points for a binary favorable outcome. Consider the sensitivity model \(d=\tau+b\), where \(\tau\) is the causal effect in percentage points and \(b\) is the remaining bias from unobserved confounding. Assume \(|b|\leq\Gamma\), where \(\Gamma\geq0\) is measured in percentage points. Since the outcome is binary, the causal effect must also lie in \([-100,100]\) percentage points.
Derive the sensitivity interval. From \(d=\tau+b\), we have \(\tau=d-b\). As \(b\) ranges from \(-\Gamma\) to \(+\Gamma\), \(d-b\) ranges from \(d-\Gamma\) to \(d+\Gamma\). Incorporating the natural range of a binary-outcome effect gives
For example, at \(\Gamma=3\) percentage points, the interval is \([1.8,7.8]\) percentage points. Under this sensitivity restriction, every compatible effect is positive. At \(\Gamma=4.8\), the interval is \([0,9.6]\) percentage points, so a strictly positive effect is no longer guaranteed. The tipping point for the conclusion “the effect is positive” is therefore \(\Gamma=4.8\) percentage points.
At \(\Gamma=105\), the untruncated bias interval would be \([-100.2,109.8]\), but these values cannot be causal effects on a binary outcome. The valid set is \([-100,100]\) percentage points. The endpoint formula with no intersection is valid only while both endpoints remain within the natural range.
This calculation is sharp within the stated model: every \(\tau\) in the intersection corresponds to a bias value \(b=4.8-\tau\) satisfying \(|b|\leq\Gamma\), and no value outside the intersection satisfies both restrictions. It does not show that confounding of a particular size is plausible; that requires substantive justification.
Worked Example: Attrition and a Threshold Tipping Point
Worked Example: Attrition and a Threshold Tipping Point
In a hypothetical evaluation, 80% of the target population respond to a follow-up survey. The mean outcome among respondents is 70 on a 0-to-100 scale. Let \(m\) be the unobserved mean among nonrespondents. Assume only that it is within \(\delta\) outcome points of the respondent mean, so \(|m-70|\leq\delta\), and that the outcome scale restricts \(m\) to \([0,100]\). Here \(\delta\geq0\).
The response fraction and respondent mean are treated as known for this calculation. The population mean is \(0.8(70)+0.2m=56+0.2m\). Combining the sensitivity restriction with the outcome scale gives
so the sharp bounds on the population mean are
For \(0\leq\delta\leq30\), the upper endpoint is \(70+0.2\delta\); for \(\delta\geq30\), it is 76, because the missing-group mean cannot exceed 100. The lower endpoint is \(70-0.2\delta\) for \(0\leq\delta\leq70\), and 56 for \(\delta\geq70\), because the missing-group mean cannot fall below zero.
At \(\delta=40\), the allowed missing-group mean is \([30,100]\), not \([30,110]\). The population mean is therefore bounded by \(56+0.2(30)=62\) and \(56+0.2(100)=76\), or \([62,76]\) points. This bound is sharp under the stated restrictions: either endpoint is attained by assigning the missing group the corresponding allowed extreme mean.
Suppose the substantive claim is that the target population mean exceeds 65 points. The claim is guaranteed by the bounds only when \(L(\delta)>65\). For \(0\leq\delta\leq70\), this condition becomes \(70-0.2\delta>65\), or \(\delta<25\). At \(\delta=25\), the lower bound equals 65, so the strict claim is no longer guaranteed. Thus 25 outcome points is the threshold tipping point for this conclusion. It is not the value at which every compatible mean falls below 65; it is where the bounds cease to establish that the mean is above 65.
Worked Example: Exclusion Violation and a Sign Tipping Point
Worked Example: Exclusion Violation and a Sign Tipping Point
Consider a hypothetical instrumental-variable analysis with a first-stage difference of \(\pi=0.40\) and a reduced-form outcome difference of \(r=8\) percentage points. Use the sensitivity model \(r=\pi\tau_C+\delta\), where \(\tau_C\) is the complier average causal effect and \(\delta\) is a constant direct effect of the instrument, measured in outcome percentage points. Assume \(|\delta|\leq\Delta\). The model retains the other assumptions needed for this reduced-form decomposition, including the stated first stage; it relaxes exclusion only through this bounded direct effect. For a binary outcome, \(\tau_C\in[-100,100]\) percentage points.
Solving for the effect gives \(\tau_C=(r-\delta)/\pi\). Since \(\pi=0.40>0\), allowing \(\delta\) to range over \([-\Delta,\Delta]\) gives
For \(\Delta=5\) percentage points, the interval is \([3/0.40,13/0.40]=[7.5,32.5]\) percentage points. A positive effect is guaranteed at this sensitivity level. At \(\Delta=8\), the interval is \([0,16/0.40]=[0,40]\), so strict positivity is no longer guaranteed. The sign tipping point is \(\Delta=8\) percentage points.
For \(\Delta=50\), the untruncated interval is \([-105,145]\) percentage points, which exceeds the possible range of a binary-outcome effect. Intersecting with \([-100,100]\) gives the valid interval \([-100,100]\). The bounds are sharp in this specified moment sensitivity model: each value in the intersection implies a direct effect \(\delta=8-0.40\tau_C\) within the allowed range. Additional restrictions on the underlying outcome distributions could narrow the set, but they are not imposed here.
Finding and Reporting a Tipping Point
The calculation begins by stating the conclusion precisely. “Robust to sensitivity” is incomplete unless it says what is being assessed. For an interval \([L(\lambda),U(\lambda)]\), typical criteria include:
- Positive effect guaranteed: \(L(\lambda)>0\). The sign conclusion tips when the lower endpoint reaches or crosses zero.
- Negative effect guaranteed: \(U(\lambda)<0\). The sign conclusion tips when the upper endpoint reaches or crosses zero.
- Effect exceeds a threshold \(c\): \(L(\lambda)>c\). The threshold claim tips when the lower endpoint reaches \(c\).
- Effect is within a range: the entire identified set must lie in that range. The conclusion tips when either endpoint leaves it.
When endpoint formulas are piecewise, first identify the relevant piece and then solve the appropriate endpoint equation or inequality. Check the resulting sensitivity value against the piece’s range, the parameter’s units, and any natural parameter bounds. For computed curves, a grid can locate a tipping point approximately; refinement around the crossing is needed if a precise value is required.
A useful report gives the baseline restriction, the meaning and units of the sensitivity parameter, the full endpoint curves or a clear description of them, and the tipping value for each stated conclusion. It should also identify which assumptions are varied and which are held fixed. Two sensitivity analyses using different parameters or restrictions need not be directly comparable merely because both report a number called a “tipping point.”
Common Pitfalls and Interpretation
- Ignoring the parameter’s natural range. Continuing a linear formula until it yields impossible effects or means produces invalid endpoints. Intersect the bound with the known support, and check whether further data restrictions apply.
- Calling any endpoint crossing a tipping point without naming the conclusion. A lower bound crossing zero changes whether positivity is guaranteed; it does not imply that the upper bound is also negative or that the effect itself has become negative.
- Confusing a sensitivity parameter with a probability. A bias limit of 5 percentage points, for example, is not a probability that bias is 5 points. It indexes a set of allowed models.
- Treating sensitivity analysis as sampling inference. These curves describe identification under specified assumptions. They do not, by themselves, account for sampling variability. Inference for bound endpoints and identified sets requires additional methods.
- Reading robustness as proof of the assumptions. A conclusion that survives a wide range of sensitivity values is robust over that range, not evidence that the model is correct or that the sensitivity range is substantively appropriate.
Check Your Understanding
Use the definitions and calculations above to answer these questions.
- In a model \(d=\tau+b\) with \(|b|\leq\Gamma\), how do you construct the interval for \(\tau\), and what additional check is needed if \(\tau\) is a binary-outcome effect?
- For the attrition example, calculate the bounds on the population mean when \(\delta=20\). Does the interval guarantee that the mean exceeds 65?
- In the instrumental-variable example, what does the tipping point of 8 percentage points say about the conclusion that the effect is positive? What does it not say?
- Why can nested sensitivity restrictions imply a nonincreasing lower endpoint and a nondecreasing upper endpoint?
- Explain why a sensitivity tipping point is not, by itself, a confidence limit or a probability statement about the assumptions.