Reporting Delta Assumptions and Tipping Results

Why Reporting Choices Matter

A tipping-point number is not interpretable on its own. A reported value such as \(\delta=-1.8\) could mean a shift of 1.8 outcome units, a log-odds shift, or something else entirely. Its direction also depends on which group and which missing values receive the adjustment. To reproduce or assess the result, a reader needs the assumptions that give delta its meaning, the reference scenario, the range examined, the analysis used to pool estimates, and the exact rule defining a “tip.”

This tutorial focuses on documenting those choices. It does not treat a delta as an empirically identified feature of the data: delta encodes a sensitivity assumption about missing outcomes that cannot be checked from the observed outcomes alone. The earlier material on Assessing the Plausibility of Extreme Delta Scenarios concerns the substantive plausibility of scenarios; reporting should make the assumptions and the basis for the chosen range visible rather than imply that the data selected a uniquely correct delta.

What a Reproducible Report Specifies

State the missing variable, the missingness pattern or group to which the adjustment applies, and the scale and direction of delta. For a continuous outcome, an additive shift might replace a reference imputation \(Y_{\mathrm{mis}}^{(0)}\) with \(Y_{\mathrm{mis}}^{(\delta)}=Y_{\mathrm{mis}}^{(0)}+\delta\); delta then has the outcome’s units. For a binary outcome, a logit shift can be written as \(\operatorname{logit}\{P(Y=1\mid X,\mathrm{mis})\}=\operatorname{logit}\{P(Y=1\mid X,\mathrm{ref})\}+\delta\). Here delta is dimensionless: the corresponding odds are multiplied by \(e^\delta\). These are different models, so a report must not describe both merely as “a shift by delta.”

Name the reference scenario and explain what \(\delta=0\) means in this analysis. Zero means no additional delta adjustment relative to the stated reference imputation model; it does not, by itself, establish that the model is true or that missingness is ignorable. Report the explored range, its increment or search method, and the substantive reasoning for its bounds. If ranges differ across groups or outcomes, specify each one separately. A range should include enough detail for a reader to see whether the tipping point lies inside the examined region or whether the analysis only establishes that it was not found there.

At each delta, describe how the imputed data were analyzed and how the resulting estimates and variances were combined. With \(M\) imputations, let \(\hat Q_j(\delta)\) be the estimate and \(U_j(\delta)\) its estimated variance in imputation \(j\). The pooled estimate is \(\bar Q(\delta)\), the mean of the \(M\) estimates. Rubin’s combining rules use the average within-imputation variance \(\bar U(\delta)\), the between-imputation variance \(B(\delta)\), and the total variance \(T(\delta)\):

Report the number of imputations and the pooling and interval procedure, including the degrees-of-freedom method when it affects the interval. State whether each delta scenario was refitted and pooled independently, or whether any quantities were held fixed. In general, estimates and variances should be recomputed under each scenario; carrying a reference-scenario standard error across a delta grid is an extra approximation, not an automatic property of delta adjustment.

Worked Examples of Reporting Decisions

Justifying the Mean-Shift Relationship

The arithmetic in the continuous example can be derived directly. Suppose \(n_T\) treatment outcomes contribute to the target mean, \(m_T\) are missing, and every missing outcome is increased by \(\delta\), while observed outcomes are unchanged. The sum of the completed treatment outcomes increases by \(m_T\delta\). Dividing by \(n_T\), the treatment mean increases by \((m_T/n_T)\delta=q_T\delta\). If the control mean is not shifted, subtracting it leaves the contrast change \(q_T\delta\). This derivation depends on a simple difference in means and a common additive shift; for regression coefficients, transformed estimands, or other patterns, the effect need not be this direct.

The underlying analysis also requires that the imputation and analysis models be specified for the intended estimand, that the delta adjustment be applied as described, and that uncertainty be pooled appropriately at each scenario. The sensitivity analysis does not make an untestable missing-outcome assumption testable. Instead, it reveals how conclusions vary across the stated alternatives. Diagnostics should therefore check that the generated values obey the outcome’s support and that the adjustment has the intended direction and magnitude. For example, an additive shift should not silently create impossible scores, and a logit shift should be interpreted on the odds scale rather than as a fixed probability increment.

Common Reporting Failures and Alternatives

A frequent mistake is to report a tipping point without defining the rule: “The tipping point was \(-1.8\)” does not tell the reader whether the estimate crossed zero, a confidence interval included zero, or a practical threshold was crossed. Another is to report only a delta range without specifying its units or the target missing values. Both errors prevent interpretation even if the numerical calculation itself is correct.

Useful diagnostics include checking the reference result against the primary analysis, inspecting estimates and interval limits across the full explored range, and confirming that the reported crossing is consistent with neighboring scenarios. Report if a tipping point was not found within the range; do not imply that it cannot occur outside it. Also disclose grid resolution or numerical tolerance, the number of imputations, and any Monte Carlo uncertainty that could materially affect the crossing. A coarse grid can locate a transition only to its resolution; a more precise-looking decimal does not overcome that limitation.

The main alternative is to report a set of prespecified scenarios rather than a single crossing. Scenario results can be easier to interpret when stakeholders can judge plausible assumptions directly, while a tipping-point summary compactly describes where a stated decision rule changes. These approaches answer different questions and can be reported together. Neither replaces the reference analysis, and neither establishes which untestable delta assumption is correct. The earlier material on Separating Statistical and Practical Tipping Points is relevant when the decision rule concerns both statistical evidence and a meaningful effect size.