Why Compare Tipping Points Across Outcomes?
A sensitivity analysis may examine several outcomes at once: for example, a clinical response, a symptom score, and a measure of functioning. Each outcome can have a different unit, a different clinically meaningful threshold, and a different response to assumptions about missing values. A raw tipping-point delta of 4 for one outcome is therefore not automatically more robust than a delta of 2 for another. The numbers may measure different kinds of departure, and the outcomes may begin at different distances from their decision thresholds.
A useful comparison starts by defining, for each outcome, the decision that matters and the sensitivity parameter that is varied. Then it expresses the tipping point on a defensible common scale, or relative to a separately justified plausible range. Standardization makes delta magnitudes more comparable; it does not make clinical thresholds interchangeable or erase differences in the outcomes’ sensitivity to missing-data assumptions.
Define the Decision Before Comparing the Numbers
Let \(\theta_j(\delta_j)\) denote the estimate for outcome \(j\) under delta setting \(\delta_j\), as obtained from the specified delta-adjusted imputation analysis. The decision rule might be that an estimated treatment benefit meets a clinical threshold, that an estimate lies on one side of a policy cutoff, or that a specified statistical criterion is met. These are different decisions: a tipping point for statistical significance is not automatically a tipping point for clinical importance.
For example, suppose larger values of \(\theta_j\) are favorable and the required clinical threshold is \(c_j\). The decision is favorable when \(\theta_j(\delta_j)\ge c_j\). If the reference estimate is favorable, the relevant tipping point is the first delta in the adverse direction at which this inequality no longer holds. If smaller values are favorable, or the decision is defined by a two-sided rule, the direction and crossing condition must be changed accordingly.
Delta values are often reported in outcome units, but those units can differ by outcome. A standardized delta expresses the assumed departure on a stated reference scale for that outcome. For illustration, write \(d_j=\delta_j/s_j\), where \(s_j\) is a prespecified, positive outcome-specific reference scale. The scale \(s_j\) must be defined consistently with the delta model and the substantive interpretation. Standardized deltas can support a comparison of departure magnitudes; they do not imply that a one-unit departure is equally plausible for every outcome.
How Scale and Threshold Determine a Tipping Point
A simple local approximation clarifies why a threshold matters. Suppose the estimate changes approximately linearly over the delta range of interest:
Here \(\beta_j\) is the change in the outcome estimate, in that outcome’s units, per one standardized delta unit. If the favorable decision requires \(\theta_j(d_j)\ge c_j\), the reference estimate is above the threshold, and adverse shifts have \(\beta_j<0\), the threshold crossing solves \(\theta_j(0)+\beta_j d_j=c_j\). Rearranging gives:
This expression shows two separate influences. A larger starting margin \(\theta_j(0)-c_j\) moves the tipping point farther away, while a greater sensitivity to delta, represented by a larger \(|\beta_j|\), moves it closer. Neither influence can be inferred from the outcome’s measurement units alone. In an actual analysis, the relationship may be nonlinear, so the tipping point should be found from the fitted sensitivity results rather than assumed to follow this approximation.
Worked Example: Two Continuous Outcomes with Different Units
Suppose a hypothetical analysis evaluates treatment benefit on two outcomes. For outcome A, the estimate is a reduction in a clinical measurement, expressed in mmHg; the decision requires a benefit of at least 5 mmHg. For outcome B, the estimate is a score improvement, expressed in points; its required threshold is 3 points. The delta analyses give the following local relationships in standardized delta units \(d\):
For outcome A, the reference margin above its threshold is \(7.2-5.0=2.2\) mmHg. Its tipping point is found by setting the estimate equal to 5 mmHg:
For outcome B, the reference margin is \(4.8-3.0=1.8\) points. Its threshold crossing is:
Both outcomes tip at 2.00 standardized delta units in this example, even though their margins and sensitivity slopes have different numerical values and units. This does not mean that a 2 mmHg shift equals a 2-point shift. It means that, under the specified standardization and sensitivity analyses, the threshold crossings occur at the same standardized delta value. The comparison is defensible only if the reference scales and delta interpretations have been defined appropriately for both outcomes.
Compare Tipping Points with Plausible Delta Ranges
A standardized tipping point describes the magnitude of departure needed to change a decision, but it does not by itself say whether that departure is plausible. Subject-matter judgment or external information may provide an outcome-specific plausible range. If the reference is zero and adverse deltas are positive, let \(B_j>0\) be the largest positive delta judged plausible for outcome \(j\). The ratio \(d_{j,\mathrm{tip}}/B_j\) locates the tipping point relative to that bound. A ratio below 1 places the crossing inside the stated range; a ratio above 1 places it beyond the range. This ratio is a comparison tool, not a probability that the decision will change.
The plausible bounds need not be numerically equal across outcomes. Their values should reflect justified assumptions about each outcome, and their derivation should be reported. A ratio comparison is informative when the bounds are elicited on sufficiently comparable grounds. If the plausibility judgments differ substantially in basis or strength, show the bounds and tipping points separately rather than presenting a ranking as objective.
Worked Example: A Threshold Crossing Inside a Plausible Range
In a hypothetical analysis, the estimated risk reduction is 0.090 at the reference delta. The prespecified policy threshold requires a reduction of at least 0.050. Sensitivity fits show that the risk reduction decreases by 0.025 per standardized delta unit over the range being considered:
The estimate and threshold are both risk differences, so their subtraction is in risk-difference units. The threshold crossing is:
Suppose subject-matter judgment specifies a plausible adverse range from 0 to 1.50 standardized delta units. The relative location is \(1.60/1.50=1.07\). Thus the tipping point is about 1.07 times the upper plausible bound: the decision remains above threshold throughout the stated range, but the crossing is close to that bound. This conclusion is conditional on the stated range and sensitivity analysis; it is not evidence that a delta of 1.60 is impossible.
If a second outcome has a tipping point of 1.80 but a plausible upper bound of 2.40, its relative location is \(1.80/2.40=0.750\). Although its raw standardized tipping point is larger, its decision changes within its stated plausible range. The two comparisons answer different questions: absolute standardized departure and robustness relative to the outcome-specific plausible range.
Multiple Sensitivity Parameters
When an outcome depends on a vector of sensitivity parameters, there may be no single tipping-point coordinate. The relevant object is a set of parameter values at which the decision boundary is reached. Comparing one coordinate at a time can be misleading when several departures act jointly. A comparison then requires a stated direction or distance rule and a clearly defined plausible region.
For a simple illustration, let \(a\) and \(b\) be two already calibrated adverse sensitivity parameters, each with plausible range from 0 to 1. Suppose outcome A has a favorable margin of 0.60 and loses that margin when \(0.40a+0.20b\) reaches 0.60. Suppose outcome B has a favorable margin of 0.50 and loses it when \(0.60a+0.10b\) reaches 0.50. One useful comparison is the smallest common fraction \(q\) of the two upper bounds needed to reach each boundary; this corresponds to setting \(a=b=q\) and increasing \(q\) from zero. For A, the boundary is \(0.40q+0.20q=0.60\), so \(q=1.00\). For B, \(0.60q+0.10q=0.50\), so \(q=0.50/0.70=0.714\). Under this explicitly stated path through the parameter space, B tips after using about 71.4% of each plausible bound, while A tips at the bounds.
This comparison depends on the chosen path and calibration. A different path, or a different distance measure, can give a different ordering. For a general tipping surface, report the decision boundaries and the plausibility region; summarize a distance only when its metric and substantive meaning are clear.
Diagnostics, Alternatives, and Limitations
Before interpreting a cross-outcome comparison, check that each outcome’s decision rule is stated in its own units and that the favorable direction is correct. Verify that the delta reference point, sign convention, standardizing scale, and plausible range are recorded for every outcome. Inspect the sensitivity results around the proposed crossing: an apparent crossing based on a coarse grid may be imprecise, and an estimate that crosses and then recrosses a threshold may have more than one crossing. In that case, report the first crossing in the specified direction and describe the additional behavior rather than treating the response as monotone.
A local slope can help explain a comparison, but it is not a substitute for checking the sensitivity curve. If the curve is nonlinear, use the evaluated delta scenarios or an appropriate root-finding method to locate the crossing, and verify the decision on either side. Also check that the compared outcomes use compatible analysis populations, estimands, and delta assumptions where the scientific question requires that compatibility. Different missingness patterns or different plausible departures may make a single joint ranking inappropriate.
The main alternatives answer related but distinct questions. Comparing raw deltas is appropriate when the deltas share units and interpretation. Comparing standardized tipping points emphasizes departure magnitude on outcome-specific reference scales. Comparing the tipping point with a plausible bound emphasizes robustness to the stated range of assumptions. Reporting each outcome’s estimate, decision threshold, curve, tipping point, and plausible range side by side is often more informative than collapsing them into one score. No summary metric establishes that one outcome is scientifically more important or that its missingness assumptions are more credible.
A common mistake is to conclude that an outcome with the larger numerical tipping point is always more robust. That reasoning ignores scale, threshold distance, sensitivity slope, and plausibility. The correction is to state what the delta measures, translate or standardize it appropriately, calculate the crossing against the outcome’s own decision threshold, and compare it with an explicitly justified plausible range. A second pitfall is to treat standardization as proof that equal delta values are equally plausible. Standardization aids scale comparison; substantive calibration is still needed.
Check Your Understanding
Answer the questions, then compare your reasoning with the solutions.
- An outcome estimate is \(12-2d\), and the favorable decision requires an estimate of at least 6. Find the standardized tipping point. If the plausible adverse bound is 2.5, where is the crossing relative to that bound?
- Why is it generally invalid to say that a raw delta of 5 for a score outcome is larger, in a meaningful sensitivity sense, than a raw delta of 2 for a blood-pressure outcome?
- Two outcomes have standardized tipping points of 1.4 and 1.8. Their plausible adverse bounds are 1.0 and 2.4, respectively. Calculate each tipping-point-to-bound ratio and identify which crosses within its stated range.
- For a two-parameter sensitivity analysis, explain why comparing only the first coordinate of two outcomes’ tipping boundaries may be misleading.
Solutions. (1) Solve \(12-2d=6\), giving \(d_{\mathrm{tip}}=6/2=3.00\). The relative location is \(3.00/2.50=1.20\), so the crossing lies beyond the stated plausible bound. (2) The deltas may have different units and meanings. A comparison requires a justified common calibration or outcome-specific standardization, not a comparison of bare numbers. (3) The ratios are \(1.4/1.0=1.40\) and \(1.8/2.4=0.750\). The first crossing is beyond its bound; the second lies within its stated range. (4) The decision can depend jointly on both coordinates. The first coordinate alone does not specify distance to the boundary or how much of the plausible parameter region is needed to reach it; the comparison needs a stated path or distance metric.