What a One-Dimensional Tipping Analysis Answers
A tipping analysis asks how a conclusion changes as an assumption about unobserved outcomes is varied. In a one-dimensional analysis, that assumption is represented by a single sensitivity value, varied while the other parts of the analysis are held fixed. The result is not a new estimate of what missing outcomes actually were. It is a map of how conclusions depend on the stated assumption.
This worked example uses a hypothetical randomized trial with a continuous outcome at 12 weeks, measured in score points, where higher scores are better. Some outcomes are missing. We vary the assumed mean among missing participants in the treatment arm, relative to a prespecified prediction, and keep the corresponding control-arm assumption fixed. The inferential conclusion of interest is whether a two-sided 95% confidence interval for the treatment-minus-control contrast lies entirely above zero.
The threshold depends on the conclusion rule. For example, the value at which a confidence interval first includes zero need not equal the value at which the point estimate itself reaches zero. A report must say which conclusion is being tracked.
Model, Assumptions, and Calculation
Let \(Y\) denote the 12-week score, and let \(\delta\) be the sensitivity parameter in score points. For each missing treatment-arm outcome, the assumed mean is the prespecified prediction plus \(\delta\). Thus, \(\delta=0\) uses the prediction unchanged; a negative \(\delta\) assumes a lower mean among those missing from the treatment arm. The assumed mean for missing control-arm outcomes remains at its prespecified prediction throughout this one-dimensional analysis.
The calculation below treats the arm sizes and observed counts as fixed. It also treats the predictions for missing outcomes and the common residual standard deviation as known, and assumes independent normal residuals with variance \(\sigma^2\) among observed outcomes. These deliberately strong assumptions make the calculation transparent; they are not automatic features of a tipping analysis. In practice, estimated predictions and variance must be handled by the chosen inferential method.
For arm \(a\), let \(n_a\) be the randomized number, \(r_a\) the number with observed outcomes, \(m_a=n_a-r_a\) the number with missing outcomes, \(\bar y_{a,\mathrm{obs}}\) the observed mean, and \(q_a\) the fixed prediction for the missing-outcome mean at the reference sensitivity value. In the treatment arm, the assumed missing mean is \(q_T+\delta\); in the control arm it is \(q_C\). The adjusted arm mean is the observed and assumed-missing contributions divided by the randomized arm size.
Under the stated variance assumptions, the missing-outcome predictions are fixed, so only the observed residual contributions contribute to the estimator’s variance. The estimated contrast’s standard error is therefore calculated here using the known \(\sigma\):
This standard error does not change with \(\delta\) under this particular model: changing \(\delta\) shifts the treatment mean by a constant but does not change the assumed residual variance. A two-sided 95% normal interval is \(\hat\Delta(\delta)\pm1.96SE\{\hat\Delta(\delta)\}\). Another analysis model may produce a sensitivity-dependent standard error or use a different interval procedure.
Worked Example: Calculate the Contrast Across a Prespecified Sequence
Suppose a hypothetical trial randomized 100 participants to each arm. In the treatment arm, 80 outcomes are observed, their mean is 8.0 points, and the prediction for the 20 missing outcomes at \(\delta=0\) is \(q_T=7.5\) points. In the control arm, 85 outcomes are observed, their mean is 7.0 points, and the prediction for the 15 missing outcomes is \(q_C=6.8\) points. Assume the common residual standard deviation is known to be \(\sigma=3.0\) points. The sequence of sensitivity values is prespecified as \(-1.5,-1.0,-0.9,-0.85,-0.5,\) and \(0\) points.
At \(\delta=0\), the treatment mean is \( \hat\mu_T(0)=\{80(8.0)+20(7.5)\}/100=(640+150)/100=7.90 \) points. The control mean is \( \hat\mu_C=\{85(7.0)+15(6.8)\}/100=(595+102)/100=6.97 \) points. Thus, the contrast is \(7.90-6.97=0.93\) points.
For any \(\delta\), the treatment mean can be simplified to \( \{640+20(7.5+\delta)\}/100=7.90+0.20\delta \). The control mean remains 6.97 points, so \( \hat\Delta(\delta)=0.93+0.20\delta \) points. In particular, each one-point decrease in the assumed missing treatment mean decreases the estimated contrast by 0.20 points, because 20% of the treatment arm has a missing outcome.
The standard error is \( 3.0\sqrt{80/100^2+85/100^2}=3.0\sqrt{0.0165}=0.3854 \) points. The 95% interval has half-width \(1.96(0.3854)=0.7554\) points. This same half-width applies at every listed value under the assumptions above.
| \(\delta\) (points) | \(\hat\Delta(\delta)\) (points) | 95% interval (points) | Interval entirely above zero? |
|---|---|---|---|
| -1.50 | 0.630 | [-0.125, 1.385] | No |
| -1.00 | 0.730 | [-0.025, 1.485] | No |
| -0.90 | 0.750 | [-0.005, 1.505] | No |
| -0.85 | 0.760 | [0.005, 1.515] | Yes |
| -0.50 | 0.830 | [0.075, 1.585] | Yes |
| 0.00 | 0.930 | [0.175, 1.685] | Yes |
The sequence brackets a change in the interval-based conclusion between \(-0.90\) and \(-0.85\) points. On the prespecified grid, the first listed value with an interval entirely above zero is \(-0.85\) points. That grid result is not itself an exact threshold; solving for the crossing gives a more precise value.
Why the Threshold Has This Value
Under this example’s assumptions, the contrast is linear in \(\delta\), and its standard error is constant. Write the half-width as \(h=1.96SE\). The interval is entirely above zero exactly when its lower endpoint is positive:
At the boundary, the lower endpoint equals zero. Solving that boundary equation identifies the continuous tipping threshold for the interval-based conclusion. This is a short derivation of the threshold, not a claim that the assumptions producing the linear contrast and fixed interval hold in every analysis.
For values greater than \(-0.873\) points, the lower endpoint is above zero; for values below it, the lower endpoint is below zero. At the boundary, the interval just touches zero, so it is not entirely above zero. The sequence values \(-0.90\) and \(-0.85\) lie on opposite sides of this boundary, consistent with their calculated intervals.
Worked Example: Report the Confidence-Interval Threshold
At the continuous threshold, the assumed mean for the missing treatment outcomes is \(q_T+\delta_{\mathrm{CI}}=7.5-0.873=6.627\) points. The treatment-arm estimate is \(7.90+0.20(-0.873)=7.725\) points, and the estimated contrast is \(7.725-6.97=0.755\) points. Its interval is approximately \(0.755\pm0.755=[0.000,1.511]\) points, subject to rounding.
A suitable report is: “Under the specified one-dimensional shift model and its variance assumptions, the 95% confidence interval for the treatment-minus-control contrast is entirely above zero for \(\delta>-0.873\) points and reaches zero at approximately \(-0.873\) points. The threshold corresponds to an assumed missing-outcome mean of about 6.63 points in the treatment arm; the control-arm missing-outcome prediction remains 6.8 points.” This gives the parameter scale, direction of movement, rule defining the threshold, and context. It does not say that \(\delta=-0.873\) is the true shift.
Different Conclusions Can Have Different Thresholds
A tipping value is tied to a decision rule, not just to the data. For the same example, the estimated contrast reaches zero when \(0.93+0.20\delta=0\), or \(\delta=-4.65\) points. That is far below the confidence-interval threshold of \(-0.873\) points. Between those values, the point estimate remains positive while its 95% interval includes zero. The estimated direction and the interval-based conclusion therefore tip at different values.
Worked Example: Find the Point-Estimate Direction Threshold
Suppose the question is whether the estimated contrast itself remains positive, rather than whether the 95% interval lies entirely above zero. The boundary is \(\hat\Delta(\delta)=0\). Substituting the fitted contrast gives \(0.93+0.20\delta=0\), so \(\delta=-0.93/0.20=-4.65\) points.
For example, at \(\delta=-1.0\), the estimated contrast is \(0.93+0.20(-1.0)=0.73\) points, still positive, but the 95% interval \([-0.025,1.485]\) includes zero. Reporting \(-4.65\) points as the threshold for “statistical evidence of a positive effect” would therefore be wrong: that value concerns the sign of the point estimate, not the interval-based conclusion. A report should name its conclusion rule explicitly.
Diagnostics, Alternatives, and Limitations
A sequence is useful only if it is sufficiently informative near the change. First calculate the estimate and interval at each prespecified value; then check whether neighboring values have different conclusions. If they do, refine the grid near that boundary or solve the boundary equation when the model permits it. Also check the arithmetic independently: the contrast should change by \(m_T\delta/n_T\) here, and the control mean should remain constant. If a reported table violates either check, the implementation or labels may be wrong.
The assumed values also need a substantive check. At the threshold in this example, the treatment-arm missing mean is about 6.63 points, compared with the observed treatment mean of 8.0 points and the fixed control missing-outcome prediction of 6.8 points. Whether that departure is plausible is a clinical or subject-matter judgment, not something established by the calculation. State the scale and reference prediction so readers can make that judgment.
A one-dimensional analysis is useful when one assumption is the focus and other assumptions can reasonably be held fixed. It is not a substitute for examining other influential assumptions. A two-dimensional analysis varies two sensitivity parameters; it can reveal that the conclusion depends jointly on assumptions that a one-dimensional slice conceals. A single imputation analysis may give a similar point-estimate calculation, but it does not automatically provide valid uncertainty for missing outcomes. Multiple imputation or model-based approaches can propagate additional uncertainty, but their interval estimates and assumptions must be specified and checked rather than borrowed from this simplified example.
The numerical standard error here is particularly restrictive: it treats predictions and \(\sigma\) as known and uses a normal interval. If predictions were estimated, their uncertainty would need to be included. If the outcome variance depended on the sensitivity shift, or if an analysis used multiple imputation, the standard error and interval could vary with \(\delta\); the threshold would then need to be found from the recalculated interval at each value. The calculation is illustrative, not a general variance formula for all tipping analyses.
A common mistake is to report only “the result tips at \(-0.873\)” without identifying what was varied or what “tips” means. Another is to treat the threshold as a discovered fact about missing participants. The correction is to state the shift’s reference and units, the fixed assumptions, the conclusion rule, the threshold’s direction, and whether it was found on a grid or by solving a boundary. A threshold summarizes sensitivity under a model; it does not verify the model or identify the missing outcomes.
Check Your Understanding
Answer each question using the hypothetical trial and assumptions in this tutorial. Short solutions follow the questions.
- What treatment-arm missing-outcome mean is assumed at \(\delta=-0.50\) points?
- At \(\delta=-0.50\), calculate the treatment contrast and decide whether the 95% interval is entirely above zero.
- Why is the confidence-interval threshold different from the point-estimate direction threshold?
- Suppose the prespecified grid had included only \(-1.0\) and \(-0.5\). What can be reported from those grid values, and what additional step supports a more precise threshold?
- Name one assumption in the example that would need to be addressed in a less idealized analysis.
Solutions. (1) The assumed mean is \(7.5+(-0.50)=7.00\) points. (2) The contrast is \(0.93+0.20(-0.50)=0.83\) points; its interval is \(0.83\pm0.7554=[0.075,1.585]\) points, so it is entirely above zero. (3) The interval rule changes when the lower endpoint reaches zero, at about \(-0.873\) points; the point estimate changes sign only when the estimate reaches zero, at \(-4.65\) points. (4) The two grid values bracket a change in the interval conclusion, but do not give its exact location; refine the grid or solve the lower-endpoint boundary equation. (5) For example, the analysis treats the missing-outcome predictions and residual standard deviation as known; an analysis estimating them must account for their uncertainty.