From a Delta Assumption to a Binary-Outcome Conclusion
A tipping-point analysis asks how strongly an assumption about unobserved outcomes must change before an analysis conclusion changes. For a binary endpoint, the change is naturally stated on the log-odds scale: a delta of zero leaves the missing-outcome prediction at its reference value, while a nonzero delta shifts the odds of success. The resulting probabilities are used to impute missing outcomes, and the treatment effect is recalculated in every completed data set.
The numerical example below uses success coded as \(Y=1\), treatment group \(T\), and control group \(C\). The effect measure is the risk difference \(RD=P(Y=1\mid T)-P(Y=1\mid C)\), expressed in percentage points when multiplied by 100. A negative value indicates a lower estimated success risk in treatment than in control; it does not by itself establish a causal effect.
Assumptions and the Chained-Equation Procedure
The reference prediction is supplied by an imputation model. For a binary endpoint, a common choice is a logistic regression conditional on treatment, baseline prognostic variables, and appropriate auxiliary variables. In chained equations, each incomplete variable is updated from its conditional model in turn; for a missing binary endpoint, the relevant update draws a value from a Bernoulli distribution with the model-predicted probability. The delta shift changes that probability for the chosen missingness pattern.
This procedure requires more than specifying a delta. The conditional models must be adequately specified for the data, the imputation process must be run long enough to produce stable draws, and the observed information must support the predictions being made. The sensitivity parameter encodes a departure from the reference missing-at-random prediction; it is not estimated from the observed outcomes. Consequently, the data alone do not identify the appropriate delta. Its direction and plausible range require substantive justification.
For a treatment-specific sensitivity analysis, one might add \(\delta\) to the linear predictor for missing treatment outcomes and use zero shift for missing control outcomes. That is a choice of sensitivity scenario, not a universal rule. If both groups may have departures, distinct shifts can be specified, but the interpretation must state which patterns and groups each shift concerns. Apply the shift once: first obtain the reference prediction from the imputation model, then add the chosen delta to its log odds.
Worked Example: Convert a Log-Odds Delta to a Probability
Suppose a hypothetical logistic imputation model gives a reference probability of success \(p_0=0.375\) for a missing treatment outcome at specified covariate values. Consider \(\delta=0.60\).
The reference odds are \(0.375/(1-0.375)=0.6000\). The delta multiplies those odds by \(\exp(0.60)\), so the adjusted odds are \((0.375/0.625)\exp(0.60)\approx1.0933\). Converting odds to probability gives $$ p_{0.60}=\frac{0.375\exp(0.60)}{0.625+0.375\exp(0.60)} \approx 0.5223. $$
Under this scenario, the predicted success probability for the missing treatment outcome is about 52.2%, compared with the reference prediction of 37.5%. The shift is conditional on the covariates in the model; it does not assert that every missing treatment outcome will be a success. Each imputation draws a binary value using the adjusted probability.
Impute, Recalculate, and Pool the Risk Estimate
For each selected delta, generate \(m\) completed data sets, retaining observed outcomes and drawing missing outcomes under the delta-adjusted chained-equation model. In each completed data set \(j\), calculate the treatment and control risks and then the risk difference \(Q_j\). Do not pool predicted probabilities first and then treat the result as if it were the completed-data treatment effect: calculate the target estimand in each completed data set.
When \(U_j\) is the estimated complete-data variance of \(Q_j\), Rubin’s rules combine the \(m\) estimates. The within-imputation variance is \(\bar{U}\), the between-imputation variance is \(B\), and the total variance \(T\) includes both sources of uncertainty:
The pooled estimate is \(\bar{Q}\), with standard error \(\sqrt{T}\). A confidence interval uses an appropriate \(t\) reference distribution and degrees of freedom for the multiple-imputation analysis; when those degrees of freedom are large, a normal critical value is a close approximation. The interval describes uncertainty under the specified delta scenario. Changing delta changes the imputed data and therefore can change the estimate, its variance, and its interval.
Worked Example: Pool a Risk Difference Across Five Imputations
Consider a separate hypothetical study with 100 participants per group. Suppose five completed data sets produce the following success counts. The table also gives the within-imputation variance using the illustrative independent-binomial estimate \(U_j=\hat p_{T,j}(1-\hat p_{T,j})/100+\hat p_{C,j}(1-\hat p_{C,j})/100\).
| Imputation \(j\) | Treatment successes | Control successes | \(Q_j\), risk difference | \(U_j\) |
|---|---|---|---|---|
| 1 | 37 of 100 | 40 of 100 | \(0.37-0.40=-0.030\) | 0.004731 |
| 2 | 38 of 100 | 41 of 100 | \(0.38-0.41=-0.030\) | 0.004775 |
| 3 | 36 of 100 | 39 of 100 | \(0.36-0.39=-0.030\) | 0.004683 |
| 4 | 39 of 100 | 40 of 100 | \(0.39-0.40=-0.010\) | 0.004779 |
| 5 | 37 of 100 | 40 of 100 | \(0.37-0.40=-0.030\) | 0.004731 |
The pooled estimate is $$ \bar Q=\frac{-0.030-0.030-0.030-0.010-0.030}{5}=-0.026. $$ The average within-imputation variance is \(\bar U=0.004740\), and the between-imputation variance is \(B=0.0000800\): four estimates differ from the mean by \(-0.004\), and one differs by \(0.016\), so \(B=[4(0.004^2)+0.016^2]/4=0.0000800\). Therefore, $$ T=0.004740+1.2(0.0000800)=0.004836,\qquad SE=\sqrt{0.004836}\approx0.06954. $$
Using a normal critical value as a large-degrees-of-freedom approximation, the 95% interval is \(-0.026\pm1.96(0.06954)\), or approximately \((-0.162,\,0.110)\). In this hypothetical analysis, the pooled estimated risk difference is \(-2.6\) percentage points, with a 95% interval from about \(-16.2\) to \(11.0\) percentage points. This interval includes zero. The variance formula here is an illustrative plug-in calculation; an applied analysis should use a complete-data variance estimator appropriate to its design and estimand.
Locate and Interpret the Tipping Point
A tipping value must be tied to a stated conclusion. A point-estimate tipping point is a delta at which the estimated effect crosses a specified threshold, often zero. An inferential tipping point is a delta at which an interval or test-based conclusion changes, such as an interval first excluding zero. These need not coincide: an estimate can cross zero while its interval continues to include zero. Report which definition is being used.
Worked Example: Find a Point-Estimate Tipping Value
In a hypothetical two-arm trial, 30 of 80 observed treatment outcomes are successes and 36 of 90 observed control outcomes are successes. There are 20 missing treatment outcomes and 10 missing control outcomes, giving 100 participants in each group. For illustration, assume the reference imputation prediction is \(p_{0,T}=0.375\) for each missing treatment outcome and \(p_{0,C}=0.40\) for each missing control outcome. Apply delta only to missing treatment outcomes.
Under delta \(\delta\), the expected treatment risk is \(0.30+0.20p_{\delta,T}\), while the expected control risk is \(0.36+0.10(0.40)=0.40\). Thus the expected risk difference is $$ RD(\delta)=0.30+0.20\frac{0.375\exp(\delta)}{0.625+0.375\exp(\delta)}-0.40. $$
For the point estimate to reach zero, the missing treatment-outcome success probability must satisfy \(0.30+0.20p_{\delta,T}=0.40\), so \(p_{\delta,T}=0.50\). The reference log odds are \(\log(0.375/0.625)=\log(0.6)\approx-0.5108\), whereas the log odds for probability 0.50 are zero. Hence the tipping value is $$ \delta_{\mathrm{tip}}=0-\log(0.6)\approx0.5108. $$
For a check, at \(\delta=0\) the expected risks are 0.375 and 0.400, so \(RD=-0.025\), or \(-2.5\) percentage points. At \(\delta=0.60\), the missing treatment probability is about 0.5223 and the treatment risk is \(0.30+0.20(0.5223)=0.4045\); the expected risk difference is \(0.4045-0.4000=0.0045\), or about 0.45 percentage points. The point estimate has crossed zero by this scenario. This calculation describes expected risks under the stated simplified predictions; actual multiple imputations will vary, and the pooled estimate must be recomputed at each delta.
Diagnostics, Comparisons, and Limitations
A defensible analysis checks the imputation and the sensitivity specification separately. Inspect chain traces and distributions of imputed binary outcomes across iterations, compare imputed and observed outcome patterns conditional on important covariates, and check that predicted probabilities are not concentrated implausibly near zero or one. Verify that the delta is applied only to the intended missingness pattern and that observed outcomes remain unchanged. Repeating the analysis with more imputations can help determine whether apparent movement across a threshold is Monte Carlo noise rather than a sensitivity result.
A delta grid is usually more informative than a single “tipping” number. Show the pooled estimate and its uncertainty across a substantively justified range, identify the reference scenario, and distinguish point-estimate crossing from inferential crossing. The earlier material on communicating tipping-point plots emphasizes this distinction visually; in a worked binary analysis, the underlying values and the exact delta interpretation should accompany the plot.
The delta approach is not the only way to handle missing binary outcomes. A reference multiple-imputation analysis under a missing-at-random assumption is the delta-zero benchmark, while complete-case analysis discards participants with missing outcomes and can be biased if their outcomes differ systematically. A pattern-mixture delta analysis makes departures from the reference prediction explicit, but it cannot determine the true departure from observed data alone. Alternative sensitivity models may encode different assumptions; none removes the need to justify the assumptions in context.
- Common mistake: treating delta as a probability difference. A delta of 0.60 does not add 0.60 to the predicted probability. It multiplies the odds by \(\exp(0.60)\), after which the odds must be converted back to a probability.
- Common mistake: adjusting observed outcomes. The shift concerns the specified missing outcomes. Altering observed outcomes too changes the estimand and is not the stated sensitivity analysis.
- Common mistake: calling a zero-crossing a significance tipping point. A point estimate can cross zero while its confidence interval still includes zero. State the criterion and recalculate pooled uncertainty at each delta.
- Failure mode: unstable or implausible predictions. Sparse data, separation, poor covariate overlap, or a misspecified logistic model can make imputed probabilities unreliable. A delta grid cannot repair a weak reference model.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A reference missing-outcome probability is 0.20. What probability corresponds to a log-odds shift of \(\log(2)\)?
- Why is a point-estimate tipping value not necessarily an inferential tipping value?
- At a particular delta, five imputed risk differences are 0.02, 0.01, 0.00, 0.01, and 0.01. What is their pooled point estimate?
- Name two checks that could reveal a problematic binary-outcome imputation analysis.
Solutions. (1) The reference odds are \(0.20/0.80=0.25\); doubling them gives 0.50, which corresponds to probability \(0.50/(1+0.50)=0.3333\). (2) The estimate may cross zero while its uncertainty interval still contains zero, so the two criteria identify different delta values. (3) The mean is \((0.02+0.01+0+0.01+0.01)/5=0.010\), or 1 percentage point; uncertainty still requires within- and between-imputation variances. (4) Examples include inspecting chain traces and checking predicted or imputed outcome distributions against observed patterns conditional on covariates.