Why Summarize Tipping Points by Outcome?
A trial may report several endpoints, and a sensitivity analysis may cause their conclusions to change at different points. One outcome might lose statistical support under a modest departure from the primary assumptions, while another remains supported over a wider range. Reporting only one number for the whole trial can hide this difference. Reporting the outcome-specific thresholds makes clear which conclusions are more sensitive, while a separate overall summary connects those thresholds to the rule used to declare trial success.
A threshold is meaningful only in relation to a specified sensitivity path and decision rule. For example, a value of 1.4 on a dimensionless sensitivity parameter is not automatically more robust than a value of 2.0 measured in outcome points. Even when the sensitivity parameter is shared, different outcomes may respond to it at different rates. The thresholds describe how conclusions change under the specified analysis; they do not measure the probability that an outcome is true or false.
For a simple one-direction path, suppose \(\delta\) starts at 0 and increasing \(\delta\) weakens evidence of benefit. If the endpoint is supported at \(\delta=0\), its tipping threshold can be written as the smallest \(\delta\geq 0\) at which it no longer meets the decision rule. This description presumes a single relevant crossing along the path. If the decision changes more than once, a single threshold does not summarize the full behavior; the crossings or the relevant region must be described instead.
Defining and Comparing Endpoint Thresholds
Before computing thresholds, specify what counts as the conclusion changing. It might be loss of a confidence interval that excludes no effect, loss of a clinically meaningful effect, or a change in effect direction. These are different decision rules and can yield different thresholds for the same endpoint. For a statistical-support rule, state the testing level and whether multiplicity adjustment is used. For a clinical rule, state the clinical margin and the direction that represents benefit.
The sensitivity scale also matters. If the same \(\delta\) has a common interpretation across endpoints—for example, a standardized departure applied along one prespecified path—the numerical thresholds can be ordered along that path. If the analyses instead use endpoint-specific sensitivity units, compare each threshold with its own plausible range or transform to a clearly defined common scale before ranking them. A numerical ranking across unlike units is not a robustness ranking.
Worked Example: Finding Two Endpoint-Specific Thresholds
Hypothetical data. Consider two benefit-oriented continuous endpoints. Along a shared, dimensionless sensitivity path, their estimated treatment effects are \(\hat{\theta}_1(\delta)=1.20-0.30\delta\) outcome points and \(\hat{\theta}_2(\delta)=0.80-0.10\delta\) outcome points. Assume the standard errors stay fixed at \(0.40\) and \(0.25\) points, respectively. For each endpoint, define support as a two-sided 95% normal confidence interval whose lower limit is above zero. For this example, do not adjust for multiplicity.
The rule for endpoint \(j\) is \(\hat{\theta}_j(\delta)-1.96SE_j>0\). At \(\delta=0\), endpoint 1 has lower limit \(1.20-1.96(0.40)=1.20-0.784=0.416\) points, and endpoint 2 has lower limit \(0.80-1.96(0.25)=0.80-0.490=0.310\) points. Both meet the stated rule.
For endpoint 1, the boundary solves \(1.20-0.30\delta-1.96(0.40)=0\). Thus \(0.30\delta=0.416\), giving \(\delta_1=0.416/0.30=1.387\). For endpoint 2, \(0.80-0.10\delta-1.96(0.25)=0\), so \(0.10\delta=0.310\), giving \(\delta_2=3.10\).
Under these assumptions and this rule, endpoint 1 tips first along the shared path. For instance, at \(\delta=1.50\), endpoint 1 has estimate \(1.20-0.30(1.50)=0.750\) points and lower limit \(0.750-0.784=-0.034\) points, so it no longer meets the rule. Endpoint 2 has estimate \(0.80-0.10(1.50)=0.650\) points and lower limit \(0.650-0.490=0.160\) points, so it still meets the rule. This ordering concerns robustness to this particular sensitivity path, not the clinical importance of either endpoint.
Multiplicity Changes the Decision Rule
When several endpoints are examined, declaring success based on any favorable result can increase the chance of at least one false positive. A multiplicity procedure changes the endpoint-level decision rule to control error across the family of tests. Because a tipping threshold is defined through a decision rule, changing that rule can change the threshold even when the sensitivity model and endpoint estimates stay the same.
For example, the Bonferroni procedure for \(m\) two-sided tests at familywise level \(\alpha\) tests each endpoint at level \(\alpha/m\). It controls the familywise error rate at no more than \(\alpha\), without requiring independence of the tests. For a normal-approximation confidence-bound rule, this replaces the unadjusted critical value with \(z_{1-\alpha/(2m)}\). Other multiplicity procedures may produce different critical values or depend on the joint distribution or testing sequence; thresholds must be computed using the procedure actually used.
Worked Example: A Bonferroni-Adjusted Threshold
Hypothetical data. Suppose a family contains three two-sided endpoint tests at familywise \(\alpha=0.05\). For one benefit-oriented endpoint, let \(\hat{\theta}(\delta)=1.10-0.20\delta\) points and \(SE=0.30\) points, assumed fixed along the sensitivity path. The endpoint is considered supported when its Bonferroni-adjusted confidence interval excludes zero in the beneficial direction. The Bonferroni critical value is approximately \(z_{1-0.05/(2\cdot3)}=z_{0.991667}=2.394\).
At \(\delta=0\), the lower bound is \(1.10-2.394(0.30)=1.10-0.7182=0.3818\) points, so the endpoint meets the adjusted rule. Its threshold solves \(1.10-0.20\delta-2.394(0.30)=0\). Therefore \(0.20\delta=0.3818\), and \(\delta=0.3818/0.20=1.909\).
Without adjustment, the 1.96 critical value would give a threshold of \([1.10-1.96(0.30)]/0.20=(1.10-0.588)/0.20=2.56\). The adjusted threshold is smaller because the familywise procedure requires stronger endpoint-level evidence. These calculations illustrate the effect of the rules; they do not establish that Bonferroni is the best procedure for every trial.
A planned testing hierarchy also affects interpretation. If endpoints are tested sequentially, an endpoint’s own nominal threshold may not be enough to preserve the trial’s overall claim: the hierarchy may require earlier tests to pass first. Report the endpoint thresholds under the actual testing procedure, and distinguish those from thresholds computed using separate, unadjusted tests. Ordering endpoints by their sensitivity thresholds is descriptive; it does not itself create a valid testing hierarchy or control multiplicity.
From Endpoint Thresholds to an Overall Conclusion
There is no unique “overall tipping point” until the trial’s success rule is stated. Suppose, for illustration, each endpoint begins supported and loses support once as \(\delta\) increases. If success requires all co-primary endpoints to be supported, the overall claim first fails when the first required endpoint tips. If success requires at least one endpoint to be supported under an appropriate multiplicity-controlled procedure, the overall claim fails only when the last eligible endpoint tips. A sequential gatekeeping rule has its own logic: an endpoint can fail to contribute to the overall claim because an earlier required test fails.
The min/max relationships follow directly from these rules. With thresholds \(\delta_1,\ldots,\delta_m\) and one-way loss of support, all endpoints are supported before \(\min_j\delta_j\); at that first threshold the all-endpoints rule fails. Under an any-endpoint rule, at least one endpoint remains supported until the greatest threshold, \(\max_j\delta_j\). These statements rely on monotone, single-crossing decisions along the same path. They need not hold if endpoints have different paths, decisions recross, or the testing rule is sequential.
Worked Example: The Overall Threshold Depends on the Claim
Hypothetical summary. Three endpoints share the same sensitivity parameter and the same prespecified multiplicity-adjusted decision procedure. Their outcome-specific thresholds are \(0.80\), \(1.40\), and \(2.60\). Assume each endpoint is supported below its threshold and unsupported above it, with no recrossing.
If the trial’s claim requires all three endpoints to be supported, its conclusion tips at \(\min(0.80,1.40,2.60)=0.80\). Once \(\delta\) reaches 0.80, the first required endpoint no longer meets the rule. If, instead, the claim is that at least one of the three endpoints is supported, the conclusion persists until \(\max(0.80,1.40,2.60)=2.60\), when the last endpoint tips.
The thresholds can be ordered as \(0.80<1.40<2.60\), so the first endpoint is least robust to increasing \(\delta\) on this path and the third is most robust. That ordering does not say which outcome is most important. Nor does the any-endpoint calculation justify an unadjusted claim: it presumes the stated multiplicity-controlled procedure is the one defining endpoint support.
Derivation and Reporting Logic
For a simple derivation, suppose a benefit-oriented endpoint has a linear sensitivity model \(\hat{\theta}_j(\delta)=a_j-b_j\delta\), where \(b_j>0\), and a fixed standard error \(s_j\). Let the decision rule require a lower confidence bound above a clinical or statistical boundary \(c_j\). With critical value \(q\), the rule is
At the boundary, equality holds. Rearranging gives \(b_j\delta=a_j-q s_j-c_j\), and hence
This expression is a valid positive threshold on the increasing-\(\delta\) path only when \(b_j>0\), the endpoint begins on the supported side, and the model and fixed-standard-error assumptions apply over the relevant range. A nonpositive numerator means the decision rule is not met at \(\delta=0\); it does not describe a future tipping point from an initially supported result. If the estimate or standard error varies nonlinearly, solve the decision boundary using the specified model rather than applying this linear formula.
Define \(\delta\), its direction, and how it changes each endpoint’s estimate or decision-relevant quantity.
Specify the clinical or statistical boundary, interval or test criterion, and any familywise adjustment or testing hierarchy.
Use the same assumptions and procedure as the reported analysis, and verify that each endpoint starts on the stated side of its boundary.
Compare thresholds on a meaningful common scale, then derive the overall conclusion from the trial’s prespecified success rule.
Diagnostics, Alternatives, and Limitations
Several checks help prevent an appealing but misleading summary. Confirm that all thresholds refer to the same sensitivity direction and comparable scale before ranking them. Verify that the baseline conclusion is actually supported under the stated decision rule. Check whether the sensitivity effects are monotone; if not, record all relevant crossings or describe the supported and unsupported regions rather than forcing a single number. Recalculate thresholds using the multiplicity procedure that governs the trial claim. Finally, assess whether the thresholds lie within a scientifically plausible sensitivity range. A threshold far outside a plausible range can be informative, but it does not show that the sensitivity assumption is realistic.
Outcome-specific thresholds are more informative than one pooled number when the endpoints answer distinct questions or have different sensitivities. A single overall threshold is still useful when it is explicitly tied to a claim such as “all co-primary outcomes must succeed.” Other summaries, such as a composite endpoint or a prioritized testing sequence, answer different questions and may change the target conclusion. They should not be substituted after seeing which endpoint tips first. Sensitivity thresholds are conditional on the model, path, and decision rule; they do not account automatically for uncertainty in estimated thresholds or establish causal effects under assumptions that the analysis does not justify.
Check Your Understanding
Answer each question using the specified path and decision rule. Brief solutions follow.
- Two endpoints have thresholds 1.1 and 2.0 on the same dimensionless path. What does their ordering say, and what does it not say?
- Three co-primary endpoints must all meet their decision rules. Their one-way thresholds are 0.6, 1.3, and 2.1. At what value does the overall claim first fail?
- A trial claim requires at least one of three eligible endpoints to meet a multiplicity-controlled rule. The one-way thresholds are 0.6, 1.3, and 2.1. When does that claim fail?
- An endpoint is supported at \(\delta=0\), but its estimate first loses support and later regains it. Is its first crossing a complete summary of its sensitivity behavior?
Solutions. (1) The endpoint with threshold 1.1 tips earlier and is less robust to increasing \(\delta\) on that path. The ordering does not rank clinical importance or support comparisons on unrelated sensitivity scales. (2) The all-endpoints claim first fails at \(\min(0.6,1.3,2.1)=0.6\), when one required endpoint tips. (3) Assuming the stated multiplicity-controlled rule and one-way loss of support, the claim fails at \(\max(0.6,1.3,2.1)=2.1\), when no endpoint remains supported. (4) No. Report the later crossing or the supported and unsupported regions; the first crossing alone omits the return to support.