What a Tipping-Point Statement Must Communicate
A tipping point is easy to misreport as a bare number: “The result tipped at −3.4.” That number is not interpretable without knowing what was varied, what the value measures, which conclusion was tracked, and how the threshold was located. A useful report lets readers reconstruct the decision being assessed and understand the scope of the sensitivity analysis.
This tutorial focuses on reporting. The analysis itself must already define a sensitivity model: a parameter or set of parameters varies an assumption about unobserved outcomes, and the chosen analysis is repeated across that variation. A tipping point is then defined relative to an explicit decision rule. For example, the rule might be that a two-sided 95% confidence interval for a treatment contrast excludes zero in the beneficial direction. A different rule, such as whether the point estimate remains above zero, produces a different tipping point and must be reported as such.
The threshold is conditional on the sensitivity model, analysis choices, and decision rule. It is not, by itself, evidence that the assumptions at the threshold are plausible or that the result is robust to every possible departure from the primary analysis.
A Reporting Sequence
A compact report can be drafted by answering six questions in order. This sequence works for a table, figure caption, methods paragraph, or results paragraph; a full report may use all of them across more than one location.
State the contrast or outcome summary and the decision rule, including the direction of the conclusion and whether it is based on the estimate, a confidence interval, or another criterion.
Give its interpretation, units or scale, and direction. If a parameter is standardized, say what standard deviation or reference quantity defines that scale.
Report the range, grid or other search procedure, and any constraints, such as varying one arm's parameter while holding another fixed.
Report its value with units and enough precision to be useful, and explain which side retains the decision and which side changes it.
Say how uncertainty in the estimated threshold or boundary was assessed, or state that the reported crossing is conditional and no uncertainty interval was calculated.
State what assumptions and parameter range the result addresses, and avoid treating the threshold as a probability or as proof that a particular missing-data scenario is realistic.
The methods description should explain how the analysis was conducted; the results statement should give the observed threshold and its meaning. If the grid is coarse, report that resolution. If the threshold was calculated by interpolation or a numerical root-finding method, state that rather than implying that the exact crossing was observed directly on the grid.
Worked Example: A One-Dimensional Results Statement
Consider a hypothetical trial with an outcome measured in score points, where higher scores are better. The analysis estimates a treatment-minus-control contrast. At the reference sensitivity setting, the estimated contrast is 3.00 points with standard error 1.10 points. For this example, assume the sensitivity parameter \(\delta\) is a downward shift, in score points, applied to the imputed outcomes for the 25% of treatment-arm participants with missing outcomes; control-arm imputations are unchanged. The analysis holds the standard error fixed across \(\delta\). These are explicit working assumptions, not general properties of sensitivity analyses.
The prespecified rule is to conclude that the contrast is positive when the lower endpoint of a two-sided 95% confidence interval is greater than zero. With the standard normal critical value 1.96, the lower endpoint at the reference setting is
Under the stated shift model, moving the missing treatment outcomes downward by \(\delta\) changes the full-arm treatment mean by \(0.25\delta\). Thus the contrast is \(3.00+0.25\delta\) points, and the lower confidence limit is
Setting the lower limit to zero locates the crossing:
Suppose the explored range was from \(-6\) to 0 points. It includes the crossing. At \(\delta=0\), the lower limit is 0.844 points, above zero. At \(\delta=-6\), it is \(3.00-1.50-2.156=-0.656\) points, below zero. The crossing is therefore consistent with both the formula and the explored range. A suitable results sentence is: “Under the specified treatment-arm shift model, the lower endpoint of the two-sided 95% confidence interval reached zero at a shift of approximately −3.38 score points among imputed treatment-arm outcomes; more negative shifts within the explored range no longer met the prespecified positive-contrast rule.”
This wording does not claim that a 3.38-point departure is likely. It also makes clear that the reported crossing depends on holding the standard error fixed and on the stated interpretation of \(\delta\).
Reporting a Region and Reporting Uncertainty
With two sensitivity parameters, a single threshold may conceal how the result depends on their joint values. Report the parameter meanings and explored ranges separately, then describe the boundary of the set that satisfies the decision rule. A plotted boundary should be described as conditional on the fitted analysis and parameterization; its location does not automatically have a confidence interval.
Worked Example: Describing a Two-Parameter Boundary
Suppose a hypothetical treatment comparison has an estimated contrast of 2.40 points at the reference setting, with standard error 0.900 points. Let \(\delta_T\) and \(\delta_C\) denote shifts, in score points, applied to imputed outcomes in the treatment and control arms, respectively. Assume the missing-outcome fractions are 0.30 and 0.20, so the contrast under the shifts is \(2.40+0.30\delta_T-0.20\delta_C\). For this example, assume the standard error remains 0.900 and use the rule that the lower endpoint of a two-sided 95% confidence interval must exceed zero.
The lower endpoint is
The boundary at which the rule changes satisfies
For example, at \((\delta_T,\delta_C)=(-2,0)\), the lower endpoint is \(0.636-0.600=0.036\) points, so the rule is still met. At \((-3,0)\), it is \(0.636-0.900=-0.264\) points, so it is not met. If the explored ranges were \(-4\) to 0 points for \(\delta_T\) and \(-2\) to 2 points for \(\delta_C\), both examples and the boundary lie within the explored region.
A concise report could state: “The positive-contrast rule was defined as a lower two-sided 95% confidence limit above zero. With treatment- and control-arm imputation shifts explored over −4 to 0 and −2 to 2 score points, respectively, the conditional boundary was \(0.30\delta_T-0.20\delta_C=-0.636\); the rule held on the side with larger values of this expression. No uncertainty interval for the boundary was calculated.” Naming the side matters: the equation alone does not tell readers which region retains the conclusion.
Uncertainty in an estimated tipping point is distinct from the confidence interval used in the decision rule. The latter determines whether the rule is met at a given sensitivity setting. An uncertainty interval for the threshold instead describes sampling uncertainty in the estimated location of the crossing, under the procedure used to calculate it. If this second quantity is reported, state the method and what was repeated. For example, a bootstrap assessment should specify whether each resample refitted the analysis, recalculated the decision statistic, and re-estimated the crossing. If a bootstrap replicate has no crossing in the searched range, the report should say how such replicates were handled rather than silently omitting them.
Worked Example: Reporting an Uncertainty Interval for a Threshold
Continue with the one-dimensional hypothetical example. Suppose 1,000 bootstrap resamples were used, and in each resample the analysis and lower confidence limit were recalculated across the specified sensitivity range. The crossing was then re-estimated. Assume the resulting estimated threshold was −3.38 points, with a percentile 95% bootstrap interval from −5.20 to −2.10 points. For illustration, suppose every resample produced an in-range crossing; this avoids needing a special rule for uncrossed resamples.
A suitable report is: “The estimated shift at which the lower two-sided 95% confidence limit reached zero was −3.38 score points. A percentile 95% bootstrap interval for the threshold was −5.20 to −2.10 points, based on 1,000 resamples in which the analysis and crossing were recalculated.” The interval’s endpoints are in the same units as the threshold. It describes uncertainty in the estimated crossing under this bootstrap procedure; it is not a range of shifts that each has a 95% probability of being the true threshold.
The bootstrap interval does not remove dependence on the sensitivity model, decision rule, or searched parameter range. Nor should it be called a confidence interval for a missing-data assumption. If the procedure does not support a calibrated interval, report that limitation and give the conditional threshold without suggesting greater precision.
Methods and Results Templates
The following templates are deliberately explicit. Replace bracketed text with the study-specific choices, and omit a claim about uncertainty if no uncertainty assessment was performed.
A methods paragraph need not repeat every numerical result, and a results paragraph need not restate the full computational recipe. But the combined report must allow readers to identify the rule, parameter scale, explored range, threshold, and uncertainty status without guessing. When a table or figure carries some details, cite its labels in the accompanying text and ensure that the caption defines units and directions.
Diagnostics, Alternatives, and Limits
Before finalizing the wording, check that the reported threshold is actually supported by the analysis. Confirm that the explored range reaches both sides of the decision rule, or state that the threshold lies beyond the range and report only a bound. Check that a coarse grid has not been described as an exact threshold, and that any interpolation or numerical search is disclosed. For a multidimensional analysis, make sure the text identifies the relevant boundary and the side where the rule holds. If the rule changes more than once, report the separate crossings or regions rather than selecting one without explanation.
The principal alternative summaries answer related but different questions. A one-number threshold is concise when there is one parameter and a clear crossing. A robustness region or boundary is more informative when conclusions depend on several parameters. A table can report selected settings and decisions, but a short table alone may not convey the location or shape of a boundary. A figure can show that geometry, but it still needs a caption specifying the decision rule, parameter scales, explored ranges, and uncertainty status. These formats can be combined; none makes the others' definitions unnecessary.
A tipping-point report has important limits. It does not validate the sensitivity model, choose a clinically plausible parameter range, or quantify the probability that the assumptions at the threshold are true. A threshold outside the explored range is not a located crossing. A threshold based on statistical significance should not be described as the point at which an effect becomes clinically meaningful: that requires a separate clinical criterion. Finally, uncertainty in the threshold can be substantial, especially when the decision statistic changes slowly with the sensitivity parameter or the crossing is near the edge of the searched range.
A common mistake is to write, “The conclusion is robust because the tipping point is −3.4,” without units, a direction, or a plausible-range comparison. The correction is to say what −3.4 measures, which decision rule it changes, what parameter range was examined, and whether the threshold falls inside a range considered plausible. Even then, describe the comparison as conditional on the chosen range, not as proof of robustness in general.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A report says, “The result tipped at −2.” Name four details needed to make this statement interpretable.
- A crossing was not reached over the explored range. What should the results statement say instead of reporting an exact threshold?
- Explain the difference between a confidence interval used to define the decision rule and an uncertainty interval for the estimated tipping point.
- A two-parameter analysis reports a boundary equation but does not say which side meets the decision rule. What is missing?
Solutions.
1. State the decision rule and direction; define what the parameter shifts and its units or scale; report the explored range; and identify which side of the threshold retains the conclusion. Also state whether uncertainty in the threshold was assessed and the key assumptions limiting interpretation.
2. Report that no crossing was observed within the explored range and give that range. If the decision rule held throughout, say so; if it failed throughout, say so. Do not extrapolate an exact threshold without a justified method and clear qualification.
3. The decision-rule interval, such as a confidence interval for the contrast at a fixed sensitivity setting, determines whether the prespecified criterion is met there. An uncertainty interval for the tipping point describes sampling uncertainty in the estimated location where that criterion changes, under a stated procedure such as a bootstrap.
4. The report must identify the side or region where the rule holds and, ideally, the region where it does not. An equation locates the boundary but does not by itself communicate the decision in either region.