Tipping Points for Effect Direction and Magnitude

Why Direction and Magnitude Need Separate Thresholds

A sensitivity analysis can ask more than whether a conclusion remains statistically significant. It can ask how much the estimated effect must change before its direction reverses, and how much before its magnitude falls below a prespecified minimum important effect. These questions concern different boundaries: zero for direction, and a nonzero threshold for magnitude. Their tipping points need not coincide.

Let \(Q(\delta)\) denote the estimate of the target effect under a specified delta-adjusted imputation scenario, with \(\delta=0\) denoting the reference scenario. The estimate may be a pooled estimate from the chosen analysis. Keep the analysis, estimand, and delta-adjustment convention fixed while evaluating this sensitivity curve; otherwise, a change in \(Q\) cannot be attributed to the delta scenario alone. The earlier material on pooled sensitivity curves gives the context for interpreting \(Q(\delta)\).

A zero boundary and a sign reversal are related but not identical. An estimate may touch zero and return to the same sign; that is a zero boundary without a direction reversal. Similarly, a magnitude boundary may be crossed more than once: an estimate can enter the region \(|Q(\delta)|<\kappa\) and later leave it. Therefore, report which crossing is being described, such as the first boundary reached from \(\delta=0\) in an increasing-\(\delta\) scenario.

Calculating Thresholds When the Curve Is Affine

For some estimands and adjustment schemes, the estimate changes linearly over the sensitivity range. In that case write \(Q(\delta)=q_0+b\delta\), where \(q_0=Q(0)\) and \(b\) is the change in the estimate per unit change in \(\delta\). The units of \(b\) are estimate-units per delta-unit. This affine form is a useful calculation model, not a universal property of delta-adjusted MICE analyses. If the analysis is nonlinear, calculate boundaries from its actual sensitivity curve rather than assuming a constant slope.

The zero-boundary formula follows by setting \(Q(\delta)=0\): \(q_0+b\delta=0\), so \(b\delta=-q_0\), and division by \(b\ne0\) gives \(\delta=-q_0/b\). This is a relevant tipping point only if it lies in the prespecified delta range and in the direction of change being examined. If \(b=0\), the estimate is constant: it has no zero boundary unless \(q_0=0\), in which case it is zero throughout the range.

For magnitude, solve \(|Q(\delta)|=\kappa\). The absolute-value equation is equivalent to the pair of equations \(Q(\delta)=\kappa\) or \(Q(\delta)=-\kappa\). For an affine curve, their candidate solutions are:

Not every candidate is a relevant crossing: check whether it lies in the allowed range and whether \(|Q(\delta)|\) actually moves from one side of \(\kappa\) to the other. When \(Q(0)>\kappa\) and the estimate decreases toward zero, the first magnitude threshold is usually the \(+\kappa\) boundary. A later \(-\kappa\) boundary may occur after the estimate has passed through zero and grown in the opposite direction.

Nonlinear Curves, Uniqueness, and First Crossings

When \(Q(\delta)\) is not affine, a boundary is found by solving \(Q(\delta)=0\) or \(|Q(\delta)|=\kappa\) on the stated range. A root is a candidate boundary; the curve on either side tells whether it is a sign reversal or a magnitude crossing. Strict monotonicity is sufficient for an equation such as \(Q(\delta)=0\) to have at most one solution on an interval, but it is not necessary: a non-monotone curve can still have exactly one root. Conversely, weak monotonicity alone does not guarantee a unique solution, because the curve may sit on the boundary over an interval. Uniqueness should be determined from the actual curve or its roots, including any boundary plateau.

For a direction boundary, a curve can also touch zero without changing sign. For instance, \(Q(\delta)=(\delta-1)^2\) has exactly one zero on \(0\le\delta\le2\), at \(\delta=1\), but is positive on both sides. That is a zero boundary, not a direction reversal. Likewise, if a weakly monotone curve equals \(\kappa\) over a range, every point in that range solves the magnitude-boundary equation; report the plateau rather than claiming a unique tipping point.

Interpretation, Diagnostics, and Alternatives

A direction tipping point describes how far the specified delta scenario must move for the point estimate to reach zero or reverse sign. A magnitude tipping point describes when that point estimate reaches the chosen minimum important magnitude. Neither threshold by itself states whether a confidence interval contains zero, whether a null hypothesis is rejected, or whether the underlying effect is clinically important. Those are distinct questions. In particular, statistical significance and practical importance do not share a boundary: a small effect may be precisely estimated, and an important-sized estimate may be imprecise.

Before interpreting a reported threshold, check the following:

• Direction and units: Verify the contrast's sign convention, the units of \(Q\) and \(\kappa\), and the meaning and scale of \(\delta\). A threshold without these is difficult to interpret.
• Range and plausibility: Confirm the solution is inside the delta range being examined and that the corresponding adjusted imputations and implied distributions remain plausible. A mathematical root outside the range is not an observed tipping point for that analysis.
• Shape and crossings: Check estimates around each candidate boundary. A non-monotone curve can have multiple roots, and a boundary root can be a touch rather than a crossing. A long or unstable plateau can also make a single-number summary misleading.
• Estimation stability: Check that apparent changes are not dominated by Monte Carlo variability in the imputations or by changes in the analysis specification. A threshold estimated from a noisy curve should not be reported with unjustified precision.

The main alternatives answer related but different questions. A confidence-interval tipping analysis tracks when an interval changes its relation to zero or another boundary, whereas direction and magnitude tipping points here track the point estimate itself. A clinical or policy threshold analysis evaluates a decision boundary defined for the application; a minimum important effect is a prespecified magnitude benchmark and should not be treated as interchangeable with that decision rule. Reporting the sensitivity curve, its relevant boundaries, and a plausible delta range is more informative than reporting a threshold alone.

A common mistake is to say, “The estimate is below the minimum important effect at the direction tipping point.” That need not be true: in the first worked example the estimate reaches the magnitude boundary at \(\delta=5.5\), but reaches zero only at \(\delta=8\). The correction is to solve the two boundary equations separately, inspect all relevant roots in the stated range, and describe each in context. Another mistake is to infer that a single threshold exists because the curve appears broadly increasing or decreasing. Check the actual crossings; monotonicity is not a necessary condition for uniqueness, and weak monotonicity can permit a boundary plateau.