Why Use Two Sensitivity Parameters?
A single tipping threshold can hide which assumptions about missing outcomes drive a change in the conclusion. In a two-arm trial, departures from a reference analysis may differ between treatment and control: missing treatment outcomes might be lower than predicted, while missing control outcomes might be higher. A two-dimensional tipping analysis examines these departures jointly. Its result is a region of parameter combinations, not one universally applicable threshold.
The example below evaluates a hypothetical trial using a confidence-interval decision. The outcome is measured in points, with higher scores representing better outcomes. The analysis asks whether the estimated treatment-minus-control difference is positive with a two-sided 95% confidence interval entirely above zero. The sensitivity parameters describe mean shifts for missing outcomes relative to their MAR-based predictions. They are assumptions for sensitivity analysis, not quantities identified from the observed data.
For the calculation to have the stated interpretation, the arm-specific shifts must refer to the same outcome scale and to the same reference analysis. In this example, each shift is a constant mean departure applied to the missing outcomes in its arm. This is a simplifying model: it does not claim that every missing participant has exactly the same departure. We also assume the MAR-based treatment-effect estimate and standard error are suitable reference quantities, and, for this worked calculation, hold the standard error fixed across the grid. If the sensitivity model changes the variance or the analysis weights, those quantities must instead be recalculated at each grid point.
Set Up the Hypothetical Trial
Suppose 100 participants were assigned to each arm. In the treatment arm, 30 outcomes are missing; in the control arm, 20 are missing. A reference analysis using MAR-based predictions estimates a treatment-minus-control mean difference of 4.00 points, with standard error 1.80 points. These are hypothetical values chosen to illustrate the calculations.
When \(\delta_T\) changes the predicted mean among the 30% of treatment outcomes that are missing, its contribution to the overall treatment-arm mean is \(0.30\delta_T\). Likewise, the control-arm contribution is \(0.20\delta_C\). Because the treatment effect subtracts the control mean, a control-arm shift enters with a minus sign. Thus the sensitivity-adjusted estimate is
This expression relies on the stated reference analysis and the missing proportions. The shifts are not multiplied by the full arm sizes; they affect the arm mean only in proportion to the fraction with missing outcomes. Units are consistent: each term on the right is in outcome points.
Worked Example: Evaluating One Pair of Shifts
Consider the assumptions \(\delta_T=-1.00\) point and \(\delta_C=1.00\) point. Missing treatment outcomes are assumed to average 1 point below their MAR predictions, while missing control outcomes average 1 point above theirs.
Substitute the values into the sensitivity-adjusted estimate:
For this example, the 95% confidence interval is calculated as the estimate plus or minus \(1.96\) standard errors. The margin of error is \(1.96(1.80)=3.528\) points, so the interval is
The lower limit is slightly below zero. Therefore this pair of assumptions does not meet the prespecified criterion that the entire 95% interval be above zero. This is not evidence that the treatment effect is negative: the point estimate remains positive, and the interval includes zero.
Construct and Classify a Grid
A grid evaluates the decision at a set of chosen values for both shifts. Here, negative treatment shifts and positive control shifts represent departures in directions that reduce the estimated treatment advantage. The table reports the lower confidence limit at each pair. An entry is marked S if that limit is strictly above zero, so the interval supports a positive difference under the specified criterion; it is marked N otherwise. “N” means the criterion is not met, not that the null hypothesis has been established.
Worked Example: Classifying a Two-Dimensional Grid
Use \(\delta_T=0,-1,-2,-3\) points for the rows and \(\delta_C=0,1,2,3,4\) points for the columns. The lower confidence limit is the adjusted estimate minus 3.528 points:
For instance, at \((\delta_T,\delta_C)=(-1,0)\), the lower limit is \(0.472+0.30(-1)-0.20(0)=0.172\) points, so the criterion is met. At \((-1,1)\), it is \(0.472-0.300-0.200=-0.028\) points, so it is not met. Applying the same calculation throughout gives:
| \(\delta_T\) (points) | \(\delta_C=0\) | \(\delta_C=1\) | \(\delta_C=2\) | \(\delta_C=3\) | \(\delta_C=4\) |
|---|---|---|---|---|---|
| 0 | 0.472, S | 0.272, S | 0.072, S | -0.128, N | -0.328, N |
| -1 | 0.172, S | -0.028, N | -0.228, N | -0.428, N | -0.628, N |
| -2 | -0.128, N | -0.328, N | -0.528, N | -0.728, N | -0.928, N |
| -3 | -0.428, N | -0.628, N | -0.828, N | -1.028, N | -1.228, N |
The numerical entries are lower confidence limits in points; the letter gives the classification. In this selected grid, the criterion is met near \((0,0)\), and fails as the assumptions become more adverse. A grid is a discrete display: a boundary can pass between its evaluated points, so the table alone does not locate the boundary exactly.
Derive and Interpret the Boundary
For the confidence interval to have its lower endpoint exactly at zero, set the lower-limit expression equal to zero. This gives the boundary between pairs that meet the positive-interval criterion and pairs that do not:
The slope is positive because a more negative treatment-arm shift can be offset, at the boundary, by a less positive or more negative control-arm shift. The criterion is met on the side satisfying \(0.472+0.30\delta_T-0.20\delta_C>0\), equivalently \(\delta_C<2.36+1.50\delta_T\). Points on the line have a lower confidence limit of exactly zero and do not meet the strict “entire interval above zero” criterion.
Worked Example: Reading the Boundary in Context
At \(\delta_T=0\), the boundary occurs at \(\delta_C=2.36+1.50(0)=2.36\) points. With no treatment-arm departure, the control-arm missing outcomes would need to average 2.36 points above their MAR predictions for the lower confidence limit to reach zero. At values of \(\delta_C\) below 2.36, the lower limit remains positive; above 2.36, it is negative.
Now suppose \(\delta_T=-1.00\). The corresponding boundary is \(\delta_C=2.36+1.50(-1.00)=0.86\) points. Thus the pair \((-1.00,0)\) lies on the side meeting the criterion, while \((-1.00,1.00)\) lies on the side not meeting it, as the grid calculations showed. The boundary describes combinations of assumptions required to change this particular confidence-interval conclusion. It is not a statement that either combination is plausible, nor does it estimate the actual missing outcomes.
Check the Analysis and Its Limits
The practical value of the two-dimensional display is that it makes the dependence of the conclusion on two arm-specific assumptions visible. It also makes clear why a one-dimensional analysis can be incomplete: fixing one shift at zero explores only a slice through the two-dimensional region. The boundary’s location depends on the chosen outcome scale, reference estimate, missing proportions, uncertainty calculation, and decision criterion.
Several checks help identify a misleading analysis:
- Check directions and signs. A positive control shift lowers the treatment-minus-control contrast in this setup, so its coefficient must be negative. If the outcome is coded so that lower values are better, the interpretation of favorable and adverse shifts must be reconsidered.
- Check units and reference values. Both shifts must be expressed in outcome points relative to the same MAR-based predictions. A shift defined relative to a different reference cannot be inserted into this equation without conversion.
- Check the uncertainty calculation. Holding \(SE=1.80\) fixed is an explicit simplifying assumption in this example. If a sensitivity analysis changes the variance estimate, degrees of freedom, or analysis weights, calculate the interval using the appropriate quantities at each grid point rather than reusing this boundary.
- Check grid resolution. A coarse grid can make a transition appear to occur at one of its grid values when it lies between values. Use a finer grid near the transition, or evaluate the derived boundary directly, while keeping the resolution adequate for the substantive scale of the shifts.
- Check whether the parameter range is meaningful. A mathematically defined region need not be scientifically plausible. Compare the shifts with justified ranges for this outcome and population; do not label the analysis robust merely because the grid omitted more adverse values.
The main alternative is a one-dimensional tipping analysis that fixes one arm’s shift and varies the other. That can answer a focused question, but it cannot show how the conclusion changes when both arms depart from their reference predictions. A two-dimensional grid shows that joint dependence, but adds choices about the ranges and spacing of two parameters. Other sensitivity analyses may use different assumptions or summarize uncertainty differently; they answer different questions and should not be treated as interchangeable.
A common mistake is to read every “N” cell as evidence of no treatment effect, or to call the boundary a threshold that the observed data have estimated. Neither is correct. The classification is only whether the stated interval criterion is met under the chosen sensitivity assumptions. The shifts are not learned from the observed outcomes, and failing to exclude zero does not establish a zero effect. Report the reference analysis, shift definitions, grid or boundary, decision rule, uncertainty method, and scientifically considered parameter ranges together.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In the hypothetical trial, calculate the adjusted estimate at \((\delta_T,\delta_C)=(-2,1)\). Does its 95% interval have a lower limit above zero?
- At \(\delta_T=-2\), calculate the value of \(\delta_C\) on the decision boundary.
- Why does a positive control-arm shift have a negative contribution to the treatment-minus-control estimate?
- What does a grid cell marked “N” establish, and what does it not establish?
Solutions. (1) The estimate is \(4.00+0.30(-2)-0.20(1)=3.20\) points. Its lower limit is \(3.20-3.528=-0.328\) points, so the interval does not have a lower limit above zero. (2) The boundary is \(\delta_C=2.36+1.50(-2)=-0.64\) points. (3) The treatment effect subtracts the control-arm mean, so increasing the control mean reduces the treatment-minus-control difference; the weighted contribution is \(-0.20\delta_C\). (4) “N” establishes only that the specified confidence-interval criterion is not met at that parameter pair under the stated calculation. It does not establish a zero or negative treatment effect, prove the null hypothesis, or show that the parameter pair is plausible.