What a Tipping-Point Contour Plot Shows
A two-parameter tipping-point analysis evaluates a prespecified statistical decision over combinations of two sensitivity parameters. A contour plot displays those combinations in a plane: one parameter is on each axis, and the plotted regions or lines summarize the decision made at each combination. Reading the plot correctly requires more than spotting a color change. The analyst must know what each axis means, which decision rule defines a change in conclusion, and how the plot marks particular scenarios.
The earlier material on tipping-point boundaries with two sensitivity parameters establishes the central idea: a boundary separates parameter combinations yielding different conclusions under a fixed analysis and decision rule. Here the emphasis is how to read that boundary, its surrounding contours, and any labeled reference scenarios. The plot describes the sensitivity analysis that was actually computed; it does not, by itself, establish which sensitivity-parameter values are scientifically plausible.
A plot may show just the boundary and two labeled regions, or it may show several contour lines for a continuous quantity such as a decision margin. These are not interchangeable. The line \(g=0\) is the decision boundary when the sign of \(g\) defines the conclusion. A line labeled \(g=0.05\), for instance, is a different level set; it is not another tipping boundary unless the decision rule specifically changes at that value. Shading can help identify regions, but the legend and axis labels—not color alone—define their meaning.
Read the Axes Before the Boundary
Start by checking the parameter names, units, tick marks, and plotted range on both axes. The horizontal coordinate gives \(\delta_1\); the vertical coordinate gives \(\delta_2\), unless the figure explicitly says otherwise. A point in the plot represents a pair of assumptions, not an individual participant or an observed outcome. In sensitivity analyses, parameter values often describe departures from an imputation model, but their signs and scales depend on the specified parameterization.
For example, if a figure defines \(\delta_1\) as a shift in the imputed outcome for one group, a positive value might shift those imputed outcomes upward. That interpretation is not universal: another parameterization could define the same sign differently. The plot’s caption or accompanying model definition must tell the reader what positive and negative values mean. Do not infer the sign convention from which region is shaded.
Likewise, do not read a plotted coordinate more precisely than its scale permits. If the axis ticks are separated by 0.1 units, visually estimating a location as 0.137 may imply unsupported precision. A contour boundary may also be drawn between computed grid points. Such a line is a visual representation or interpolation, not necessarily a result calculated at every point on the line.
Worked Example: Identifying the Conclusion Region
Consider a hypothetical analysis with two sensitivity parameters measured in standardized outcome units. The analysis defines positive values of \(\delta_1\) and \(\delta_2\) as shifts that make the treatment contrast less favorable. Its signed decision margin is \(g(\delta_1,\delta_2)=0.12-0.8\delta_1-0.5\delta_2\). The prespecified conclusion is “favorable” when \(g>0\), “not favorable” when \(g<0\), and exactly on the decision boundary when \(g=0\). What can be concluded at the reference point \((0,0)\) and at \((0.10,0.10)\)?
At the reference point, substitute both coordinates into the margin:
Because \(0.12>0\), the point \((0,0)\) is in the favorable-conclusion region. At the second point,
Because \(-0.01<0\), \((0.10,0.10)\) is in the not-favorable region. In a contour plot of this margin, the boundary is \(0.8\delta_1+0.5\delta_2=0.12\). The example’s sign convention is specific to this hypothetical analysis; the same visual direction cannot be assumed for another analysis.
Locate and Interpret the Decision Boundary
For a plot with two conclusion regions, first identify the boundary line or curve separating them. Then locate the parameter pair of interest and determine which side contains it. If the figure labels the regions, use those labels to state the conclusion. If it gives only a signed margin, the sign convention must be stated before classifying points. A point on the boundary is a threshold case under the specified decision rule; it is not automatically evidence for either neighboring conclusion.
A simple derivation shows how a boundary coordinate can be read when the margin is linear. For the margin in the example, setting \(g=0\) gives
At \(\delta_1=0.10\), the boundary height is therefore
At that coordinate, \(g=0.12-0.08-0.04=0\), confirming the boundary calculation. At the same \(\delta_1\), a value \(\delta_2=0.10\) lies beyond the boundary in the direction that reduces \(g\), while \(\delta_2=0.05\) lies on the other side. This algebra is a useful check on a visual reading when the model or a plotted margin is available.
For a curved boundary or a plot that only reports a few computed coordinates, a local estimate may be made by linear interpolation, but it should be labeled as an estimate. Suppose the plotted boundary crosses \((0.10,0.08)\) and \((0.05,0.16)\). If the boundary is approximately straight between these points, then at \(\delta_1=0.075\), halfway between 0.10 and 0.05, the interpolated boundary coordinate is
The interpolation estimates the location of the boundary between those plotted points; it does not prove that the true boundary is straight there. Curvature, sparse computation, or a poorly resolved plot can make that estimate inaccurate.
Worked Example: Reading a Scenario Near a Boundary
Use the same hypothetical linear analysis. A plot labels the boundary crossings \((0.10,0.08)\) and \((0.05,0.16)\), and marks a scenario \(R\) at \((0.075,0.10)\). Estimate the boundary height at the scenario’s \(\delta_1\), then check which conclusion applies at \(R\).
The scenario’s first coordinate is halfway between 0.10 and 0.05, so linear interpolation gives a boundary height of
The scenario has \(\delta_2=0.10\), which is below the estimated boundary height. Under this model, increasing \(\delta_2\) reduces \(g\), so a point below the boundary is on the favorable side. Check directly:
The positive margin confirms that \(R\) is in the favorable region. The direct calculation is stronger than relying on visual position alone. Since the interpolated boundary is approximate, the small positive margin also signals that the point is close to the threshold; it should not be described as far from tipping.
Read Contour Labels and Reference Scenarios
A reference scenario is a named or marked parameter pair selected for interpretation, such as the analysis baseline or a prespecified alternative. Its label does not guarantee that it lies in either conclusion region. Read its coordinates from the axes, check whether the marker is placed at the correct location, and then compare it with the decision boundary. If the reference scenario is defined in a caption or legend, use that definition rather than assuming that “reference” always means \((0,0)\).
A plot can also display labeled contours of a continuous quantity. Each contour connects parameter pairs that yield the same value of that quantity. For a margin \(g\), the \(g=0\) contour is the decision boundary under the sign-based rule above, while contours such as \(g=0.05\) and \(g=-0.05\) show equal-margin locations on either side. If the plotted quantity is instead an estimate, test statistic, or another measure, its contours must be interpreted according to that quantity’s label and units. A contour label alone does not tell you whether the statistical conclusion changes there.
Worked Example: Checking Labeled Reference Scenarios
Suppose a second hypothetical plot uses parameters \(a\) and \(b\), and its signed decision margin is \(g(a,b)=0.10-0.4a-0.3b-0.2ab\). The plot labels \((0,0)\) as the reference scenario, \(P=(0.10,0.10)\), and \(Q=(0.20,0.20)\). The rule is favorable for \(g>0\). Classify the three scenarios and verify the boundary at \(a=0.10\).
At the reference scenario,
Thus, the reference scenario is in the favorable region. At \(P\),
Because \(0.028>0\), \(P\) is also favorable. At \(Q\),
Therefore \(Q\) is in the not-favorable region. To locate the boundary at \(a=0.10\), set the margin to zero and solve for \(b\):
Substitution verifies the result: \(g(0.10,0.1875)=0.06-0.32(0.1875)=0\). At \(a=0.10\), \(P\), with \(b=0.10\), lies below this boundary and has a positive margin. This example also shows why a reader should not assume every boundary is a straight line: the interaction term \(ab\) makes the boundary depend nonlinearly on \(a\).
Assumptions, Diagnostics, and Limitations
A contour plot supports an interpretation only under the analysis and plotting conditions that produced it. At minimum, the outcome, estimand, sensitivity-parameter definitions, and decision rule should be held fixed across the plotted combinations. The calculation at each pair must apply those definitions consistently. If the displayed conclusion uses a threshold on a computed statistic, that threshold must also remain the same throughout the plot. If any of these change across locations, the apparent boundary may combine different analyses rather than show a single sensitivity analysis.
Several checks help identify a misleading or internally inconsistent display:
- Check axis direction and units. Confirm which parameter is horizontal and which is vertical, and whether values are raw units, standardized units, or another scale.
- Check the legend and boundary label. Make sure the line identified as the decision boundary is the level where the specified conclusion changes, not merely a contour of another quantity.
- Check the reference marker’s coordinates. A misplaced marker or swapped axis can make a correct scenario appear to support the wrong conclusion.
- Check points on both sides. If available, compare the computed decision at nearby parameter pairs on either side of the displayed line. A sign change in the signed margin should agree with the plotted region labels.
- Check edge and disconnected regions carefully. A boundary that reaches the edge of the displayed range may continue beyond it. Separate favorable regions can also occur; do not assume there is only one crossing or one connected region.
- Check precision near the line. If a point is close to the boundary, use the underlying calculation or a clearly qualified interpolation rather than relying on a rounded visual reading.
The contour plot is a deterministic summary of the computed analysis over its displayed parameter values. It is not a probability distribution over sensitivity parameters, and distance from the line is not automatically a measure of statistical certainty or scientific plausibility. A plot cannot determine whether the parameter range is credible; that judgment depends on substantive knowledge and the stated sensitivity assumptions. Conclusions also should not be extrapolated outside the axes’ displayed range without additional calculations.
The main alternative presentations each have different strengths. A table of selected parameter pairs gives exact values and is useful for named scenarios, but can hide the shape of the boundary between rows. One-parameter slices show how the decision changes as one parameter varies while the other is held fixed, but do not summarize the full two-dimensional pattern. A contour plot makes the joint structure easier to see, but may make exact coordinates harder to read and can conceal details when the grid is sparse. Use the plot for the overall geometry, and use calculations or a table to verify consequential scenario classifications.
Common Mistake
A common error is to say, “The marker is in the blue area, so this scenario is robust,” without checking the legend, axes, or decision rule. The color may indicate the opposite conclusion, may represent a contour level rather than a conclusion region, or may be affected by a reversed sign convention. Even when the color is correctly understood, one point in a favorable region does not show that all nearby or plausible parameter combinations are favorable.
The correction is to name the scenario’s coordinates, state the plot’s sign or region convention, and compare the point with the decision boundary. For a point near the boundary, verify the classification with the underlying signed margin or decision calculation. This gives a context-specific statement—for example, “At \((0.075,0.10)\), the calculated margin is \(0.01\), so the prespecified favorable conclusion holds”—instead of an unsupported claim based on color or visual distance.
Check Your Understanding
Answer each question using the plot-reading principles above. Solutions follow the questions.
- A plot defines \(g>0\) as favorable and shows a point with \(g=-0.03\). What conclusion applies at that point?
- A reference marker is at \((0.05,0.10)\), while the boundary at \(\delta_1=0.05\) is at \(\delta_2=0.12\). If increasing \(\delta_2\) reduces the signed margin, is the marker on the favorable or not-favorable side?
- Two labeled contours are \(g=0.04\) and \(g=0\). Under a rule that changes conclusion at \(g=0\), which contour is the decision boundary?
- Why is a conclusion based only on a marker’s color insufficient when the legend and parameter definitions have not been checked?
Solutions. (1) Since \(-0.03<0\), the point is in the not-favorable region. (2) The marker is below the boundary, on the side with the larger margin when increasing \(\delta_2\) reduces that margin; it is favorable if the stated boundary separates the positive-margin side below from the negative-margin side above, as in the linear example. The direction must be confirmed from the sign convention or a computed value; the boundary coordinate alone does not establish it. (3) The \(g=0\) contour is the decision boundary; \(g=0.04\) is another equal-margin contour. (4) Color may encode a different quantity, and the sign convention and axis meanings determine which conclusion the point represents.