Decision Rules That Determine a Tipping Point

Why the Decision Rule Matters

A sensitivity analysis traces how an estimate and its uncertainty change as an unobserved-data assumption changes. But a curve alone does not determine a tipping point. The point depends on which decision is being tracked: rejecting a null hypothesis, meeting a confidence-interval requirement, or crossing a prespecified threshold can yield different boundaries on the very same curve.

This tutorial focuses on that dependence. The sensitivity parameter is denoted by \(\delta\), and its allowed range is denoted by \(\mathcal D\). At each value of \(\delta\), the analysis produces an estimate \(\hat\theta(\delta)\) of a target quantity \(\theta\), and possibly a standard error \(SE(\delta)\). A tipping point is meaningful only after a rule, the direction of interest, and the parameter range have been stated.

For example, a rule might be “reject \(H_0:\theta=0\) using a two-sided test at level \(\alpha=0.05\).” Another might be “the lower endpoint of a 95% confidence interval must exceed 1 unit.” A third might be “the point estimate must be at least 2.5 units.” These are distinct rules, even when applied to identical sensitivity-adjusted estimates.

Conditions for Locating a Boundary by Equality

Consider a two-sided normal-approximation test of \(H_0:\theta=0\) against \(H_a:\theta\ne 0\). Assume that the estimate is approximately normal under the analysis model, that \(SE(\delta)>0\), and that the reported standard error is appropriate for the estimate at each \(\delta\). The test statistic is \(Z(\delta)=\hat\theta(\delta)/SE(\delta)\). The rule rejects when \(|Z(\delta)|>z_{1-\alpha/2}\), where \(z_{1-\alpha/2}\) is the standard normal quantile.

If the estimate is positive throughout the portion of the path being examined, this rule reduces to \(\hat\theta(\delta)/SE(\delta)>z_{1-\alpha/2}\). If both functions are continuous and a boundary is reached by crossing the cutoff, the boundary satisfies \(\hat\theta(\delta)=z_{1-\alpha/2}SE(\delta)\). Continuity matters: without it, the decision can change in a jump without the standardized estimate ever equalling the cutoff. If the estimate changes sign, the relevant boundary may instead be on the negative side, where \(\hat\theta(\delta)=-z_{1-\alpha/2}SE(\delta)\). It is therefore unsafe to solve only the positive-side equation without checking the path.

When the estimate is affine in \(\delta\) and the standard error is constant, the positive-side equation has a simple solution. Suppose \(\hat\theta(\delta)=a-b\delta\), with \(b>0\), and \(SE(\delta)=s>0\). Provided the solution lies in the specified range and the estimate is positive there, the rejection boundary is

This calculation locates an equality boundary only under the stated conditions. Monotonicity of the test statistic on the examined path supports a unique crossing. If the path is not monotone, there may be more than one change in decision; if the path is discrete or discontinuous, inspect the rule’s value at the available points rather than assuming an equality solution exists.

Confidence-Interval Rules and Test Rules

A confidence interval gives another way to state a decision criterion, but some interval rules are exactly equivalent to a test rule. For the same normal approximation, a two-sided level-\(\alpha\) test of \(H_0:\theta=0\) rejects precisely when the matching \(100(1-\alpha)\%\) interval excludes zero. In the positive-estimate region, this means the interval’s lower endpoint is above zero. Thus a 95% interval-excludes-zero rule and a two-sided test at \(\alpha=0.05\) do not produce different tipping points when they use the same estimate, standard error, and approximation.

A different interval requirement can, however, have a different boundary. For example, requiring the lower endpoint of a 95% interval to exceed a prespecified value \(c\), rather than zero, defines a different decision. For a positive estimate and constant standard error \(s\), the interval is \(\hat\theta(\delta)\pm1.96s\), and the requirement is \(\hat\theta(\delta)-1.96s>c\). The boundary equality is appropriate for a continuous path that crosses this threshold. It is the chosen interval requirement—not the phrase “confidence interval” by itself—that determines the result.

Prespecified Decision Criteria

A decision rule need not be based on a test or an interval. It may be a threshold set in advance for an estimate or another reported quantity. For instance, a protocol might require the estimated effect to remain at least 2.5 units. Such a criterion answers a different question from statistical significance. The threshold should be specified before examining the sensitivity results; selecting it afterward to obtain a desired conclusion makes the resulting boundary difficult to interpret.

When an Equality Is Not the Boundary

The equality calculations above rely on a continuous crossing. Sensitivity analyses may instead be evaluated only on a grid, or a model update may make the path discontinuous. In either case, the decision can change between available values without an observed equality. The correct target is then the first change in the decision along the specified ordering of \(\delta\), not a root that may not exist.

Practical Checks and Common Pitfalls

Before reporting a tipping point, state the rule in a form that can be evaluated at every \(\delta\). Then check the estimate, uncertainty calculation, parameter range, direction of movement, and boundary convention. A test based on a normal approximation, for example, requires a suitable approximation and a standard error appropriate to each sensitivity setting. A confidence-interval rule requires the specified confidence level and endpoint construction to be stated. A prespecified threshold requires a clear scale and units.

Monotonicity also matters. If the decision rule holds, then fails, then holds again as \(\delta\) changes, a single boundary does not summarize the full path. Check the decision across the range before describing one tipping point as unique. For a grid, report resolution honestly: if the last value satisfying the rule is 1.2 and the first failing value is 1.3, the grid locates the change between those values, not necessarily at an exactly known value.

• Pitfall: treating every boundary as a test cutoff. A confidence-interval threshold or a minimum-estimate criterion need not use the test’s critical value. Write down and solve the actual rule.
• Pitfall: assuming matched tests and intervals differ. Under the same two-sided normal procedure, a level-\(\alpha\) test of zero and the matching interval’s exclusion of zero are equivalent. A different result signals a different rule or an inconsistent calculation.
• Pitfall: solving an equality on a jump. A discontinuous or grid-based path can change classification without attaining the cutoff. Find the change by direct evaluation.
• Pitfall: reporting a unique threshold without checking the path. A boundary equality alone does not show that the rule holds on one entire side and fails on the other. Check values on both sides and inspect for additional changes.
• Pitfall: interpreting a statistical decision as a substantive guarantee. Failure to reject is not proof of no effect, and satisfying a point-estimate threshold does not establish statistical significance.

The key comparison is not “test versus interval” in the abstract. It is between explicitly stated rules applied to the same sensitivity path. Equivalent rules should agree; rules with different thresholds can yield different tipping points. When equality is not justified, report the observed decision change rather than an assumed root.