Annotating Tipping-Point Figures

What an Annotated Tipping-Point Figure Must Show

A tipping-point figure is useful only if a reader can tell what is being varied and what conclusion changes. Its annotation should make the sensitivity parameters, decision rule, tipping boundary, and regions on either side interpretable without relying on the analysis code or an accompanying table. The reference analysis also needs its own clear marker: it gives readers a starting point for judging how the conclusion changes under alternative sensitivity scenarios.

This tutorial focuses on making those elements visible and mathematically consistent. A line or contour is not self-explanatory: its meaning depends on the chosen outcome, scale, decision rule, and parameterization. A report-ready figure states those choices in the axes, labels, legend, and caption.

Specify the Decision Rule Before Labeling the Plot

Begin by stating exactly what conclusion the figure tracks. For example, an analysis may classify a result as statistically significant in a specified direction when the lower endpoint of a two-sided 95% confidence interval for a treatment contrast is above zero. Another analysis may track only whether the estimated contrast is above zero. These are different rules and can produce different boundaries. A label such as “favors treatment” is therefore incomplete unless the figure or caption says whether it refers to an estimate, an interval-based criterion, or another specified rule.

Let \(u\) and \(v\) denote two sensitivity parameters, and let \(\hat{\Delta}(u,v)\) and \(SE(u,v)\) be the estimated contrast and its standard error under those settings. Under a normal-approximation two-sided 95% confidence interval, the lower endpoint is \(L(u,v)=\hat{\Delta}(u,v)-1.96SE(u,v)\). If the chosen rule is \(L(u,v)>0\), the boundary consists of settings satisfying \(L(u,v)=0\). One side meets the rule and the other does not. The figure must label which side is which; the boundary itself is not a region where the strict rule is met.

This construction assumes the stated confidence-interval approximation is appropriate for the analysis, and that the estimate and standard error have been calculated for each displayed sensitivity setting. If the decision rule instead concerns the sign of the estimate, use \(\hat{\Delta}(u,v)=0\) as the boundary. Do not reuse an interval-based label for an estimate-based boundary.

A Short Derivation: From the Rule to the Boundary

The boundary is obtained by setting the quantity in the decision rule equal to its cutoff. This is the threshold at which the conclusion changes, provided the displayed parameter range crosses it and the rule changes across it. For the interval-based rule above, set the lower confidence limit to zero:

For illustration, suppose the analysis gives \(\hat{\Delta}(u,v)=1.20-0.040u-0.030v\) points and \(SE(u,v)=0.25\) points throughout the plotted range. Then:

Setting this equal to zero gives \(0.040u+0.030v=0.710\), or \(v=23.67-1.333u\). The region below this line has \(L(u,v)>0\) and meets the stated rule; the region above it has \(L(u,v)<0\) and does not. This algebra justifies both the line and the region labels. In a real analysis, if the standard error changes with the sensitivity parameters, use the calculated \(SE(u,v)\) at each setting rather than assuming a constant value.

Build a Two-Parameter Figure and Verify Its Labels

For two parameters, the plot typically uses one sensitivity parameter on each axis and represents the decision outcome by regions separated by a line or contour. Units and direction matter. A label such as “Parameter 1” does not tell readers what assumption is changing; a label should use the parameter’s defined name and unit or scale. State whether increasing a parameter represents a more adverse, more favorable, or otherwise different assumption when that direction is not obvious from its definition.

If a plot shades regions, make the legend explain the shading in terms of the decision rule. If it uses a line type or color for the boundary, identify that line independently of the shading. A brief caption should name the outcome or contrast, the rule, the confidence level if relevant, and the meaning and units of the sensitivity parameters. If a displayed grid approximates a boundary, say so; do not imply greater numerical precision than the calculation supports.

Reference Markers, Scenarios, and Figure Diagnostics

A reference marker and a sensitivity scenario are not interchangeable. The reference marker represents the baseline parameter settings used for the reference analysis, often the origin when parameters are defined as departures from that baseline. Alternative scenarios are other points or paths in the sensitivity-parameter space. A marker at the origin should not automatically be called “observed data”: it is the reference analysis at the specified settings, not a claim that all outcomes used in that analysis were observed.

When selected sensitivity scenarios are shown, give them a distinct symbol or a clearly described line style. Label points with short identifiers and define those identifiers in the legend or caption. Avoid using the same visual mark for the reference analysis and a scenario that happens to share a plotted value. When several scenarios are crowded near the boundary, a compact legend or a small table can identify them more clearly than overlapping text labels.

Before release, check the figure against the analysis output. Useful checks include:

• Confirm that the plotted axes use the intended parameter definitions, units, and directions.
• Substitute at least one point on each side into the decision rule and verify the region labels.
• Check that points marked as exactly on the boundary satisfy the equality defining it, within the calculation’s numerical tolerance.
• Confirm that the reference marker is at the actual baseline settings and that the legend describes it accurately.
• Check that the boundary and labels remain distinguishable in grayscale and are not obscured by markers, grid lines, or overlapping text.

These checks can reveal a sign reversal, swapped axes, or a region label copied from the wrong side of a contour. They also help detect a plot whose displayed boundary comes from an interpolated grid but whose label suggests an exact analytic solution. If the classification changes abruptly across adjacent grid cells, inspect the underlying calculations and grid resolution rather than smoothing away the discrepancy.

Comparisons and Limitations

A figure complements, rather than replaces, a tipping-point table. A table can report selected parameter values and precise calculated results; a figure makes the shape of a boundary and the arrangement of regions easier to see. A single threshold is compact when one parameter varies, while a two-parameter figure can display interactions in the decision surface. Neither format removes the need to state the decision rule and parameter definitions.

The figure also does not establish whether the sensitivity scenarios are plausible. It displays consequences conditional on the assumptions represented by the parameters. Plausibility ranges or reference scenarios should be identified separately and described accurately; a boundary crossing outside a relevant range has a different practical interpretation from one within it. A plotted boundary alone is not an estimate of how likely the assumptions are.

Other limitations should be visible in the reporting. A boundary may depend on a chosen confidence-interval rule, model, scale, and parameter range. If the standard error or other analysis components vary with the sensitivity settings, the boundary need not be linear. A plot restricted to a small range can hide additional crossings, while a coarse grid can miss narrow features. State the plotted range and calculation method, and avoid extrapolating the displayed line or contour beyond the range evaluated.

Common Mistake

A frequent error is to label one side “robust” and the other “not robust” without defining robust, or to put the observed-data label on whichever point seems closest to the data. Those labels obscure both the decision criterion and the meaning of the reference settings. The correction is to name the exact rule in the region label or caption, and to identify the reference analysis by its baseline parameter settings. If the figure tracks an interval-based conclusion, say so explicitly; if it tracks the sign of an estimate, say that instead.

Another error is to infer the region label from the apparent slope of a line. The correct side follows from the inequality in the decision rule, not from whether it appears visually “above” or “below.” Evaluate the rule at a known point on each side, as in the examples, before finalizing the labels.