Statistical and Clinical Tipping Points

Two Different Decisions Can Tip at Different Values

A sensitivity analysis asks how conclusions change when an assumption about missing outcomes changes. The same sensitivity scenario can affect two distinct questions: is the estimated effect statistically distinguishable from the null value, and is the estimated effect large enough to matter clinically? A value of the sensitivity parameter that changes the answer to one question need not change the answer to the other.

To keep these questions separate, define the decision rule for each before locating a tipping point. Statistical significance is determined by an inferential procedure, such as a hypothesis test at a specified \(\alpha\). Clinical importance is determined by a meaningful-effect threshold and a stated rule for comparing the estimate, or its uncertainty interval, with that threshold. The threshold and rule should be justified for the outcome and decision context; they are not supplied by the p-value.

This definition concerns conclusions across sensitivity scenarios. It does not say that a tipping point is a probability, a directly observed characteristic of participants, or a boundary between assumptions that are true and false. It is a boundary in the analysis produced by a specified model and decision rule.

State Both Rules Before Calculating

For illustration, consider an effect measured in outcome points, with larger positive values representing greater improvement on the outcome in the stated application. Suppose the statistical rule is a two-sided test of \(H_0:\theta=0\) at \(\alpha=0.05\), using a normal approximation. When the estimate is positive, the rule rejects \(H_0\) if \(\hat{\theta}(\delta)/SE(\delta)>1.96\). Equivalently, the corresponding 95% confidence interval excludes zero on the positive side.

Now suppose the clinical rule is that the estimated improvement must be at least \(M\) points. This is a point-estimate rule: the clinical conclusion is favorable when \(\hat{\theta}(\delta)\geq M\). Another decision-maker might instead require the lower confidence limit to exceed \(M\). That is a different, more uncertainty-sensitive rule and will generally produce a different clinical tipping point. Neither rule should be silently substituted for the other.

The analysis must also be suitable at each sensitivity value: the estimand and outcome scale should remain the same, the method used to calculate \(\hat{\theta}(\delta)\) and \(SE(\delta)\) should be appropriate, and the inferential calibration should be credible. For a clinical tipping point, \(M\) must be expressed on the same scale as \(\hat{\theta}(\delta)\) and supported by clinical or decision-making context. Sensitivity scenarios should be interpreted within the range considered plausible; a mathematical crossing outside that range may have little practical relevance.

Why the Boundaries Usually Differ

The statistical boundary depends on both the estimate and its uncertainty. In the positive-estimate case with a normal approximation, the boundary for a two-sided 5% test is \(\hat{\theta}(\delta)=1.96SE(\delta)\). By contrast, under the point-estimate clinical rule, the boundary is \(\hat{\theta}(\delta)=M\). These equations have different right-hand sides, so even if both decisions use the same estimate, their crossing values need not coincide.

For example, if the standard error is fixed and \(M>1.96SE\), a positive estimate can fall below the clinical threshold while remaining statistically significant. If \(M<1.96SE\), the ordering can go the other way. If the standard error changes with \(\delta\), the statistical boundary also moves, while a fixed clinical threshold does not. This comparison is a derivation of the distinction, not a guarantee that either boundary exists in the sensitivity range: the estimate may never cross one of them.

When the Standard Error Changes Too

A common mistake is to solve only for where the estimate crosses a fixed critical value. That works only when the relevant standard error is fixed and the stated test applies. In a sensitivity analysis, the estimate and its uncertainty may both vary with \(\delta\), so the statistical boundary must use \(SE(\delta)\) at the crossing. The following example shows the calculation when uncertainty changes.

The Clinical Rule Can Include Uncertainty

A clinical decision does not have to be based on the point estimate alone. For example, a policy may require the lower endpoint of a 95% confidence interval to exceed \(M\) before concluding that a clinically important benefit is supported. This is more demanding than requiring only \(\hat{\theta}(\delta)\geq M\). It answers a different question: whether the data and model support an effect above the meaningful threshold with the specified confidence-interval criterion.

Diagnostics, Alternatives, and Limitations

Before interpreting either tipping point, check that the calculations correspond to the intended decision. Recompute the estimate and uncertainty under each sensitivity scenario using the specified analysis; confirm that the outcome scale, estimand, test, confidence level, and clinical threshold have not changed inadvertently. Near a boundary, verify the arithmetic using adjacent sensitivity values, and avoid reporting more numerical precision than the analysis supports.

• Check the inferential rule. A test and its confidence interval should agree when they are constructed as matching procedures. If they do not, investigate whether different standard errors, degrees of freedom, sidedness, or confidence levels were used.
• Check the clinical threshold. Confirm its units and rationale. A threshold on one outcome scale cannot be compared directly with an estimate on another scale without an appropriate, explicit conversion.
• Check how uncertainty changes. If \(SE(\delta)\) varies, use its scenario-specific value. Treating it as constant can place the statistical tipping point incorrectly.
• Check the sensitivity range. A computed boundary outside the plausible range being examined is not evidence that a decision changes under plausible assumptions.
• Check the model output. Implausible predictions, unstable estimates, or a poorly supported reference model can make a precise-looking boundary misleading.

The main alternative to reporting two separate tipping points is to report a single decision rule that combines statistical and clinical requirements, such as requiring both statistical significance and an estimate above \(M\), or requiring a confidence bound above \(M\). Such a rule may suit a particular decision, but it does not make the two concepts interchangeable. State which conditions are required and report each result so readers can see what drives the combined decision.

Tipping points also inherit limitations from the chosen sensitivity model, the range of \(\delta\), and the uncertainty calculation. They summarize where a specified conclusion changes; they do not validate the missing-data assumptions, establish that a scenario is plausible, or prove the clinical value of an outcome difference. A single reported value can also conceal uncertainty in the estimated boundary or changes in the shape of the estimate across scenarios. Interpret it alongside the sensitivity results and the assumptions that generated them.

Common Mistake

Wrong reasoning: “The result is statistically significant, so the benefit is clinically meaningful,” or, conversely, “the estimate exceeds the clinical threshold, so the study has demonstrated a statistically significant effect.” Both statements merge separate criteria. Statistical significance depends on the estimate relative to its uncertainty and a null hypothesis. Clinical importance depends on a meaningful threshold and a stated clinical decision rule.

Correction: Report the two conclusions separately. For example: “At this sensitivity value, the estimate remains statistically significant under the prespecified test, but it falls below the 4-point point-estimate threshold.” This wording identifies the rule behind each conclusion without treating either as proof of the other.