Worked Tipping-Point Analysis for a Continuous Treatment Effect

Why a Continuous-Outcome Tipping Point Needs a Defined Criterion

A sensitivity analysis is easier to interpret when it answers a specific question: how different would the unobserved outcomes have to be for the substantive conclusion to change? For a continuous treatment effect, that answer depends on the direction and units of the outcome, the delta adjustment, the pooled uncertainty, and the criterion used to define “change.” A clinical threshold and a test of whether the effect differs from zero are not interchangeable.

Consider a hypothetical randomized trial measuring a symptom score at 12 weeks, where lower scores are better. Define the treatment effect as the mean score in the treatment arm minus the mean score in the control arm. Negative values therefore favor treatment. Suppose clinicians define a mean reduction of at least 2 points as clinically important. In this tutorial, the main tipping criterion is that the upper endpoint of the two-sided 95% confidence interval is below −2 points. This is a deliberately demanding criterion: the interval must support a benefit exceeding the clinical threshold, not merely an average benefit or an effect different from zero.

The reference imputation model is assumed to be appropriate under its stated missing-at-random (MAR) condition, conditional on the variables included in that model. Delta adjustment represents departures from that reference for the missing outcomes. It does not establish that a particular delta is correct, nor does observed data identify the distribution of outcomes that were not observed. The analysis therefore depends on making the adjustment’s direction, target, and plausible range explicit.

Set Up the Estimand and the Pooling Calculation

Let \(Q_j(\delta)\) be the treatment-minus-control estimate from completed dataset \(j\) under delta scenario \(\delta\), for \(j=1,\ldots,m\). Let \(U_j(\delta)\) be its estimated variance within that completed dataset. Rubin’s rules combine these quantities as follows:

Here \(B\) measures variation between the completed-data estimates, while \(T\) combines that variation with average within-imputation variance. The pooled standard error is \(SE(\delta)=\sqrt{T(\delta)}\). A confidence interval is formed as \(\bar{Q}(\delta)\pm t_{0.975,df}SE(\delta)\), using an appropriate multiple-imputation degrees-of-freedom calculation. Each delta scenario requires pooling; the estimate and its uncertainty should not be treated as if they came from one completed dataset.

A useful special case explains the shape of the sensitivity curve. Suppose the treatment-arm estimand is an unadjusted mean, a proportion \(q\) of treatment-arm outcomes is missing, and every imputed missing treatment outcome is shifted by the same \(\delta\). The treatment-arm mean increases by \(q\delta\), because the shift applies to \(q\) of its observations; the control mean is unchanged. Thus:

This is a linear relationship for the point estimate under these conditions. It does not guarantee that \(T(\delta)\), the degrees of freedom, or the confidence interval width stays constant. Those quantities can change when completed-data estimates or variances change. In an adjusted regression analysis, the estimate’s response to delta may also depend on the model and covariates. Refit the analysis and pool at each delta rather than assuming a linear curve for every estimand.

Construct the Sensitivity Curve and Locate the Tipping Point

For this hypothetical trial, suppose 40% of treatment-arm outcomes are missing, so \(q=0.400\), and use the simple arm-specific mean setup above. A positive delta makes missing treatment outcomes worse (higher symptom scores) than their reference imputations. It therefore moves the treatment-minus-control estimate toward zero. For the worked curve, suppose the completed-data variance calculations yield the same \(\bar{U}=0.6400\), \(B=0.02000\), and \(df=3062\) at each displayed delta. This stability is an assumption of this numerical illustration, not a general property of delta adjustment. The pooled standard error and interval margin are consequently unchanged at approximately 0.8149 and 1.598 points.

The calculation is a special-case derivation of the tipping point: the upper endpoint is \(\bar{Q}(0)+q\delta+c\), where \(c\) is the fixed confidence-interval margin. Solving \(\bar{Q}(0)+q\delta+c=-2\) gives \(\delta=\{-2-\bar{Q}(0)-c\}/q\). In realistic analyses, the interval margin may vary with delta. Then the tipping point is found by evaluating the pooled criterion across a sufficiently fine grid, locating where it changes status, and refining the search. A reported numerical threshold should reflect the precision supported by that search and by Monte Carlo error.

Assumptions, Diagnostics, and Alternative Summaries

A credible worked analysis needs more than correct arithmetic. Check that the treatment contrast, favorable direction, outcome units, missingness pattern, and set of shifted imputations all match the scientific question. The reference MICE models should include variables needed for the analysis and relevant predictors of missingness and outcome. The delta must represent a departure from those reference imputations—not a second application of a shift already incorporated elsewhere. The trial’s randomized assignment supports a treatment comparison, but interpretation as an intention-to-treat causal effect also relies on the trial’s design and analysis conditions, including consistency and no interference.

At each delta, inspect the completed outcomes and the pooled results. Warning signs include implausible values, a shift in the wrong direction, an apparent jump in the curve where the specified adjustment should change smoothly, unstable estimates across imputations, or an interval width that changes unexpectedly. For an unadjusted mean with a common fixed shift, the point estimate should obey \(\bar{Q}(\delta)-\bar{Q}(0)=q\delta\); a discrepancy signals a coding, denominator, or estimand mismatch. This identity does not apply automatically to adjusted estimands or nonlinear summaries. Also assess whether Monte Carlo error is small relative to the reported tipping-point precision.

The main alternatives answer different questions. A reference MAR analysis reports the result under the reference imputation model but does not assess departures from it. A collection of prespecified delta scenarios compares selected assumptions without necessarily identifying a threshold. A tipping-point curve identifies where a stated decision rule changes, but the threshold is only as informative as the criterion and plausible delta range. A statistical tipping point based on whether a confidence interval includes zero is not the same as this tutorial’s clinical tipping point at −2. The introductory material on sensitivity analysis provides the broader context; here, the emphasis is the calculation and interpretation for a continuous treatment effect.

A tipping point is not a probability that a delta is true, and it does not validate the missing-outcome assumptions. It can also be misleading to report only a single threshold: readers need the direction of delta, which outcomes it modifies, the clinical rule, the pooled uncertainty method, and whether plausible scenarios reach that threshold. If the criterion already fails at the reference scenario, there may be no positive delta “tipping” from success to failure; the relevant question may instead be what assumptions restore the criterion. If the curve is nonmonotone, report that behavior rather than forcing a single threshold.

Common Pitfall