Why Tipping-Point Plots Need Careful Design
A tipping-point plot is a communication device for a sensitivity analysis, not merely a picture of model output. Its purpose is to show how an analysis result changes under specified assumptions about missing values, and to identify where a stated conclusion changes. A reader should be able to tell what each sensitivity parameter means, which scenario is the reference, what rule defines a “tip,” and how uncertain the estimated result is.
The plot cannot establish that a particular delta assumption is correct or plausible. It displays consequences conditional on the assumptions used to produce it. Clear labels and annotations help readers distinguish those consequences from evidence about which assumptions should be believed.
Before plotting, specify the estimand, the meaning and units of each delta parameter, the range of scenarios evaluated, and the conclusion rule. For example, the estimand might be a treatment contrast in outcome units, while the rule might be that its confidence interval excludes zero. A different rule—such as crossing a clinically meaningful threshold—can produce a different tipping point. Statistical and practical thresholds should not be presented as if they were interchangeable.
What a Reader Should See
A useful sensitivity curve makes the scenario and the conclusion legible at a glance. Label the horizontal axis with the delta parameter’s name and units, including what a positive or negative change means for the imputed values. Label the vertical axis with the estimand and its units. If delta is a shift on an outcome scale, say so; “delta” alone is not an adequate axis label. State the evaluated range and identify the reference scenario, often \(\delta=0\), using a line, marker, or concise annotation.
Include a horizontal line for the relevant threshold. If the inferential rule depends on a confidence interval, show the estimate and its uncertainty as well as the threshold. A vertical line or label can mark the estimated tipping point, but it should be clear whether this is where the point estimate crosses a threshold or where an inferential rule changes. Those locations may differ.
For a two-parameter analysis, label both axes with parameter meanings and units, mark the reference pair, and explain which side of the contour meets the stated rule. A contour without a clearly identified decision region is difficult to interpret. If the analysis evaluates only a finite grid, avoid drawing a smooth boundary that suggests more precision than the grid and calculation support. State whether the displayed boundary is calculated, interpolated for display, or only sketched as a guide.
| Plot element | What to communicate | Common ambiguity to avoid |
|---|---|---|
| Axes | Parameter names, meanings, and units | Showing a delta value without explaining its scale |
| Reference | The reference scenario and why it is relevant | Letting readers assume the reference is known to be true |
| Threshold | The numerical cutoff and decision rule | Using “tipping point” without defining what tips |
| Uncertainty | What the intervals or bands represent | Presenting a pointwise interval as simultaneous coverage |
| Decision region | Which side satisfies the rule | Relying on colour alone to encode the conclusion |
Worked Examples
Worked Example: Marking an Inferential Tipping Point on a Curve
Suppose a hypothetical sensitivity analysis estimates a treatment contrast \(Q\), measured in outcome points, at several values of a delta shift measured in outcome points. The prespecified inferential rule is that a 95% confidence interval for \(Q\) excludes zero in the positive direction. The estimates and standard errors below are hypothetical and have already been obtained using the analysis and pooling procedure appropriate to the study.
| \(\delta\) (outcome points) | \(\hat{Q}(\delta)\) (outcome points) | \(SE(\hat{Q})\) (outcome points) |
|---|---|---|
| 0 | 2.40 | 0.900 |
| 1 | 1.60 | 0.800 |
| 2 | 0.800 | 0.750 |
For illustration, use the normal-based interval \(\hat{Q}\pm 1.96SE\). At \(\delta=1\), the lower endpoint is
The upper endpoint is \(1.60+1.96(0.800)=3.168\) outcome points, so this interval lies just above zero. At \(\delta=2\), the lower endpoint is
The inferential rule therefore changes between the two evaluated scenarios. For a graphic, plot the estimates, show their 95% intervals, draw a horizontal zero line, and distinguish the reference at \(\delta=0\). A straight-line interpolation of the lower endpoints would place the crossing at approximately
This is an approximate interpolated location, not a newly evaluated model result. The plot should label it accordingly, or show only that the crossing lies between \(\delta=1\) and \(\delta=2\). The point estimate itself does not cross zero in this displayed range: at \(\delta=2\), it remains \(0.800\) outcome points. The plot must therefore avoid labeling the confidence-interval crossing as a point-estimate crossing.
Worked Example: Reading a Two-Parameter Tipping Contour
Consider a hypothetical analysis with two delta parameters, \(\delta_A\) and \(\delta_B\), each measured in outcome points. Suppose the estimated contrast over the plotted region is described by \(\hat{Q}=3.00-1.20\delta_A-0.800\delta_B\), with standard error \(0.750\) outcome points. Use the illustrative normal-based 95% interval and define the positive-conclusion region as the scenarios where its lower endpoint exceeds zero.
The lower endpoint is \(\hat{Q}-1.96(0.750)\). Since \(1.96(0.750)=1.470\), the decision rule becomes
The tipping contour is the equality boundary \(1.20\delta_A+0.800\delta_B=1.530\). For instance, at \(\delta_A=0\), the boundary occurs at \(\delta_B=1.530/0.800=1.913\) outcome points. At \(\delta_A=1.00\), it occurs at \(\delta_B=(1.530-1.20)/0.800=0.4125\) outcome points. At the reference pair \((0,0)\), the lower endpoint is \(3.00-1.470=1.530\), so the reference scenario is in the positive-conclusion region. A plot should label the boundary, mark \((0,0)\), and identify the side satisfying the inequality. These calculations assume the stated estimate and standard error formula apply throughout the displayed region; if the actual standard error changes with the deltas, the contour must instead be calculated from the scenario-specific uncertainty.
Worked Example: Reporting Uncertainty Without Overstating It
Suppose a hypothetical analysis evaluates the inferential tipping point repeatedly under a procedure designed to quantify its sampling uncertainty. The resulting estimated tipping-point values, in units of the delta parameter, have a median of \(1.10\) and a central 95% interval from \(0.60\) to \(1.80\). The plotted sensitivity curve also shows pointwise 95% confidence intervals for \(\hat{Q}(\delta)\) at each displayed delta value.
A careful caption could state: “The vertical marker shows the estimated inferential tipping point at \(\delta=1.10\); the horizontal range from \(0.60\) to \(1.80\) shows the central 95% interval from the tipping-point uncertainty procedure. The shaded bands are pointwise 95% confidence intervals for the estimand at each evaluated delta.” These are two distinct uncertainty summaries: one describes uncertainty in the estimated tipping-point location, and the other describes uncertainty in the estimand at fixed delta values.
The pointwise intervals do not, by themselves, provide 95% simultaneous coverage of the whole curve. Nor does the interval for the tipping point mean there is a 95% probability that the fixed, true tipping point lies in that particular interval. The caption should name the procedure and its interpretation, and the graphic should use separate visual encodings for the curve intervals and tipping-point interval.
Deriving and Displaying the Decision Boundary
A plot’s threshold should follow from the stated decision rule, not from the appearance of the curve. Let \(\hat{Q}(\boldsymbol{\delta})\) be the estimated quantity for sensitivity-parameter vector \(\boldsymbol{\delta}\), and let \(SE(\boldsymbol{\delta})\) be its standard error. For a positive conclusion based on a normal-based 95% interval lying above a threshold \(T\), the lower endpoint must exceed \(T\):
The tipping boundary is where equality holds. This follows directly because the conclusion changes when the lower endpoint moves from above \(T\) to at or below \(T\). If the decision rule instead concerns the point estimate, replace the lower endpoint with \(\hat{Q}(\boldsymbol{\delta})\); that produces a different boundary in general. A practical threshold may also differ from zero, and should be explicitly labeled with its units and rationale.
The normal critical value is appropriate only when the analysis justifies a normal-based interval. If inference uses a \(t\)-critical value or another interval procedure, use the corresponding rule and state it. Likewise, estimates and standard errors should be generated consistently for each delta scenario, with the same estimand and analysis definition. A curve can look smooth even when estimates are noisy or only available at a sparse grid, so visual smoothness is not evidence that the underlying relationship is smooth.
Diagnostics, Alternatives, and Limitations
Check the plot against the underlying scenario results before presenting it. Confirm that the reference marker corresponds to the intended delta values, axis units and directions are correct, and the plotted intervals match the reported standard errors and interval method. Recalculate any annotated tipping location from the stated rule. For contours, verify that a test point on each side does and does not meet the rule, respectively. If the contour is based on a grid, disclose the grid resolution or interpolation method when it could affect the apparent boundary.
Sensitivity curves are usually clearer than a table when readers need to see direction, shape, and threshold crossings across a one-dimensional range. Tables are useful for exact values at selected scenarios, especially when precise numbers matter more than visual trends. For two parameters, a contour plot communicates a boundary and its regions compactly; a table or heat map can be preferable when there are few scenarios or when the response is irregular. A plot should complement, not replace, a concise report of the assumptions and important numerical results.
Common failure modes include cropping the delta range so the reference or crossing disappears; using an unlabeled delta axis; omitting the threshold; using colour without line styles or labels; and drawing a precise-looking contour from a coarse grid. Another error is to call any change in statistical significance a change in practical importance. The correction is to state the rule explicitly, distinguish inferential from practical thresholds, and display uncertainty in a way that identifies exactly what it represents.
Finally, no annotation can eliminate dependence on the chosen sensitivity model, delta range, or decision rule. A tipping plot is conditional evidence about how conclusions respond to specified assumptions; it does not validate those assumptions. Include a short caption or accompanying text describing the delta interpretation, reference scenario, estimand, threshold, interval method, and any interpolation or uncertainty procedure. This information lets readers assess the graphic without mistaking a visual summary for a complete sensitivity analysis.
Check Your Understanding
Answer each question using the plot-design principles above. Brief solutions follow.
- A curve shows an estimated contrast against delta, but the horizontal axis is labeled only “delta.” What essential information is missing?
- A point estimate remains positive while its confidence interval begins to include zero. Is this a point-estimate tipping point or an inferential tipping point?
- For the contour example, determine whether \((\delta_A,\delta_B)=(0.500,1.00)\) satisfies the positive-conclusion rule.
- Why should a plot label a band as pointwise if each interval is calculated separately at each delta value?
- A contour is drawn smoothly through a grid of only four evaluated scenarios. What should the analyst disclose or change?
Solutions. (1) The axis needs the parameter’s meaning, scale, and units, including what positive values represent. (2) It is an inferential tipping point because the interval-based decision changes while the point estimate remains positive. (3) The weighted sum is \(1.20(0.500)+0.800(1.00)=1.40\), which is less than \(1.530\); therefore the lower endpoint exceeds zero and the scenario satisfies the rule. (4) Separate pointwise intervals do not automatically provide simultaneous coverage of the entire curve, so calling them a general 95% band would overstate what they show. (5) Disclose that the boundary is interpolated from a coarse grid, avoid implying unsupported precision, and consider showing the evaluated points or calculating the rule on a finer grid.