Delta Adjustments and Tipping-Point Analysis

Why Adjust Imputed Outcomes?

An imputation model for missing outcomes necessarily makes assumptions about outcomes that were not observed. A common reference model predicts missing outcomes from observed data under a missing-at-random assumption: conditional on variables in the model, the distribution of the missing outcomes matches the distribution the model predicts from observed outcomes. If missingness still depends on the unobserved outcome after conditioning on those variables, that reference model can be wrong in a systematic way.

A delta adjustment makes that departure explicit. It shifts the imputed outcomes in a chosen direction and by a chosen amount, while leaving observed outcomes unchanged. The shift can differ across treatment arms or missingness patterns. Recomputing the causal effect over a range of shifts shows how much the conclusion depends on assumptions about the unobserved outcomes. The shift is a sensitivity parameter, not something identified from the observed outcomes alone.

The causal estimand and the assumptions used to identify it from observed outcomes are distinct from the assumptions about missing outcomes. For example, a randomized study may identify an intention-to-treat effect in the absence of missing outcomes, but outcome missingness can still make the observed-data analysis depend on a missingness model. Delta analysis probes that dependence; it does not by itself correct confounding or make an otherwise unidentified causal effect identified.

Definition and Sign Convention

Definition: A delta adjustment adds a specified shift \(\delta_{a r}\) to outcomes imputed for individuals in treatment arm \(a\) and missingness pattern \(r\), relative to a reference imputation model. The shift is expressed in the outcome's units. A positive delta means the missing outcomes are assumed to be higher than the reference model predicts; a negative delta means they are assumed to be lower. Observed outcomes are not shifted. A tipping point is a value, or boundary of values, at which a specified conclusion changes as the delta parameters vary.

For a continuous outcome, if an individual's reference-model imputed value is \(Y^{\mathrm{MAR}}\), the delta-adjusted value is

Y^{\mathrm{imp}}(\delta)=Y^{\mathrm{MAR}}+\delta_{a r}.

Here \(a\) identifies treatment arm and \(r\) identifies the missingness pattern. A pattern can distinguish, for example, individuals with a missing final outcome but an observed earlier outcome from individuals with both outcomes missing. Which patterns are useful depends on the study's data structure and the sensitivity question. The delta values are chosen by the analyst to represent departures worth examining; the observed data do not generally determine them.

The reference imputation model should be appropriate for the analysis: it should include relevant treatment, outcome predictors, and covariates, and its imputations should be compatible with the causal estimand and effect estimator. Delta adjustment supplements that model; it does not repair a poorly specified reference model. The usual causal assumptions, such as randomization in a trial or exchangeability and positivity in an observational analysis, remain relevant. The sensitivity analysis addresses the additional uncertainty created by missing outcomes.

How Shifts Change an Estimated Effect

For a mean-difference estimand, let \(\hat\mu_{a r}^{\mathrm{MAR}}\) be the reference-imputation mean for arm \(a\) and pattern \(r\), and let \(w_{a r}\) be that pattern's proportion in arm \(a\). The weights sum to one within each arm. If the pattern-specific shifts are constant within each pattern, the adjusted arm mean is

\hat\mu_a(\boldsymbol{\delta}) =\sum_r w_{a r}\left(\hat\mu_{a r}^{\mathrm{MAR}}+\delta_{a r}\right).

For a treatment-minus-control effect, \(\hat\tau(\boldsymbol{\delta})=\hat\mu_1(\boldsymbol{\delta})-\hat\mu_0(\boldsymbol{\delta})\). Therefore,

\hat\tau(\boldsymbol{\delta}) =\hat\tau(\boldsymbol{0}) +\sum_r w_{1r}\delta_{1r} -\sum_r w_{0r}\delta_{0r}.

This formula shows why the same numerical shift need not have the same impact in both arms: its contribution is also multiplied by the proportion in the affected pattern. In regression, standardization, or a more complex multiple-imputation analysis, the adjusted estimate may not reduce to this simple weighted-mean expression. The general procedure is still to apply the shifts to the missing-outcome imputations, carry out the prespecified causal analysis, and record the resulting estimate and uncertainty.

Worked Example: Shifts in Two Treatment Arms

Hypothetical data. Consider a randomized study with 100 people per arm and a continuous outcome measured in points. In the treatment arm, 80 outcomes are observed with mean 8.0 points, and the 20 missing outcomes have a reference-imputation mean of 7.0 points. In the control arm, 90 outcomes are observed with mean 6.0 points, and the 10 missing outcomes have a reference-imputation mean of 5.5 points. Suppose the target is the treatment-minus-control mean difference.

Under the reference imputations, the completed-data means are

\hat\mu_1(\boldsymbol{0}) =\frac{80(8.0)+20(7.0)}{100} =\frac{640+140}{100} =7.80\ \text{points},

\hat\mu_0(\boldsymbol{0}) =\frac{90(6.0)+10(5.5)}{100} =\frac{540+55}{100} =5.95\ \text{points}.

Thus \(\hat\tau(\boldsymbol{0})=7.80-5.95=1.85\) points. Now suppose the analyst considers missing treatment-arm outcomes to average 2 points below their reference imputations and missing control-arm outcomes to average 1 point above theirs. These choices are \(\delta_1=-2\) points and \(\delta_0=+1\) point. Only the missing outcomes shift, so

\hat\mu_1 =\frac{80(8.0)+20(7.0-2)}{100} =\frac{640+100}{100} =7.40\ \text{points},

\hat\mu_0 =\frac{90(6.0)+10(5.5+1)}{100} =\frac{540+65}{100} =6.05\ \text{points}.

The adjusted effect is \(7.40-6.05=1.35\) points. Equivalently, the change from the reference estimate is \(0.20(-2)-0.10(1)=-0.50\) points, giving \(1.85-0.50=1.35\) points. This is a sensitivity result for the stated shifts; it does not establish that either shift is the true departure from the reference model.

Pattern-Specific Adjustments and Tipping Points

A single delta per arm can hide important differences among missingness patterns. If, for example, people with only a final outcome missing are thought to differ less from the reference model than people who missed both an intermediate and final assessment, separate deltas can represent those assumptions. The relevant pattern proportions determine how strongly each shift contributes to the arm mean.

A tipping point requires a clearly stated conclusion to track. Possible conclusions include whether an effect is positive, whether it exceeds a prespecified clinically important threshold, or whether a confidence interval excludes zero. These are different criteria and can have different tipping points. For one varying delta, solve for the value at which the criterion's boundary is reached. With several deltas, the boundary is generally a set of combinations rather than a single number. A one-dimensional analysis then needs to specify a path through that set, such as shifting both arms by fixed proportions of one common parameter.

Worked Example: Separate Shifts for Two Missingness Patterns

Hypothetical data. Suppose a reference analysis estimates a treatment-minus-control effect of 1.60 points. In the treatment arm, 12% of participants have pattern 1 and 8% have pattern 2; the remaining participants have observed outcomes. In the control arm, the corresponding proportions are 5% and 5%. Assume pattern 1 and pattern 2 have separate deltas within each arm. Consider the values \(\delta_{11}=-1\), \(\delta_{12}=-3\), \(\delta_{01}=+0.5\), and \(\delta_{02}=+1\), all in outcome points.

The treatment mean changes by \(0.12(-1)+0.08(-3)=-0.12-0.24=-0.36\) points. The control mean changes by \(0.05(0.5)+0.05(1)=0.025+0.050=0.075\) points. Because the effect subtracts the control mean, its change is \(-0.36-0.075=-0.435\) points. Hence the adjusted effect is

\hat\tau(\boldsymbol{\delta}) =1.60+0.12(-1)+0.08(-3)-0.05(0.5)-0.05(1) =1.165\ \text{points}.

Suppose the prespecified substantive criterion is an effect greater than 1.0 point. Under these particular deltas, the estimate remains above that threshold by \(1.165-1.0=0.165\) points. This calculation assesses only the stated combination. It does not show that every combination with similar average shifts would preserve the conclusion, since shifts in different patterns receive different weights.

A Tipping Point for an Inferential Conclusion

When the conclusion concerns a confidence interval, the estimate and its uncertainty both matter. In a full analysis, the imputations and causal estimator should be recomputed at each delta setting, and uncertainty should be estimated using the procedure appropriate to that analysis, including the variation among imputations. The following example uses an explicitly simplifying assumption of a fixed standard error to show how to locate a threshold. It is not a general rule that the standard error remains fixed as deltas change.

Worked Example: Locating a Two-Sided Confidence-Interval Threshold

Hypothetical sensitivity path. Suppose a treatment-effect estimate in points varies with a single delta according to \(\hat\tau(\delta)=2.00+0.30\delta\), where delta is measured in outcome points. The coefficient 0.30 represents the proportion of the relevant observations affected by this shift in this simplified mean analysis. Suppose, for illustration, that the standard error is held fixed at 0.45 points and a 95% normal-approximation interval is used.

The interval's margin of error is \(1.96(0.45)=0.882\) points. At \(\delta=0\), the estimate is 2.00 points and the interval is \([2.00-0.882,\ 2.00+0.882]=[1.118,\ 2.882]\) points, which excludes zero.

For any estimate \(\hat\tau(\delta)\), this two-sided interval excludes zero when \(|\hat\tau(\delta)|>0.882\). Along the path of decreasing estimates from 2.00 points, its first loss of exclusion occurs at the positive boundary:

2.00+0.30\delta=0.882 \quad\Longrightarrow\quad \delta=\frac{0.882-2.00}{0.30}=-3.727\ \text{points}.

At this value, the interval just reaches zero. For example, at \(\delta=-5\), the estimate is \(2.00+0.30(-5)=0.50\) points, and the interval is \([0.50-0.882,\ 0.50+0.882]=[-0.382,\ 1.382]\) points, which includes zero. But the interval does not exclude zero for every still more negative delta. At \(\delta=-10\), the estimate is \(2.00+0.30(-10)=-1.00\) point, and the interval is \([-1.00-0.882,\ -1.00+0.882]=[-1.882,\ -0.118]\) points. It excludes zero again, now on the negative side. The estimate crosses zero at \(\delta=-2.00/0.30=-6.667\), and the interval excludes zero again once the estimate is below \(-0.882\), at \(\delta=(-0.882-2.00)/0.30=-9.607\). Thus the first loss of exclusion along the decreasing path is not a claim that all more negative values yield intervals containing zero.

How to Conduct and Report the Analysis

Specify the reference analysis.
State the causal estimand, the reference imputation model, and the estimator. Make clear which assumptions underlie the causal analysis and which missing-outcome assumptions the sensitivity analysis will vary.

Define the patterns and delta scale.
Identify the treatment arms and missingness patterns that receive separate shifts. State the sign convention and outcome units so a value such as \(-2\) has a direct interpretation.

Choose a sensitivity range or path.
Use subject-matter reasoning to choose values worth examining. With several deltas, state how they vary together; a single tipping value is meaningful only relative to such a path or a specified criterion boundary.

Recompute the causal analysis.
Shift only the imputed missing outcomes, then calculate the effect and its uncertainty at each setting. Do not assume the estimate or standard error responds in the same way in every estimator.

Report the threshold in context.
Describe which conclusion changes, at what delta value or combination, and in which direction. Report the outcome units and clarify that the threshold is a sensitivity result, not an estimate of the true missing outcomes.

Interpretation and Common Pitfalls

Delta adjustments matter because a causal conclusion can look stable under the reference imputation model but change under plausible departures from it. A useful report presents estimates across a range of deltas, not only the value at which a preferred conclusion holds. For multiple deltas, a table or contour display can show how the result depends jointly on treatment-arm and pattern-specific assumptions. The analysis is informative even when it cannot establish which delta is correct: it reveals what assumptions are needed for a stated conclusion.

Shifting observed outcomes. Delta adjustment changes the imputed values for missing outcomes; it does not alter recorded outcomes.
Ignoring the direction of the contrast. For a treatment-minus-control effect, a downward shift in treatment outcomes lowers the effect, while a downward shift in control outcomes raises it. The signs follow from subtracting the control mean.
Calling the delta data-identified. A tipping value describes where a conclusion changes under a specified model and path. It does not show that the missing outcomes actually differ from the reference predictions by that amount.
Using “the interval excludes zero” as a one-sided condition. For a two-sided interval centered at an estimate with fixed margin \(c\), exclusion occurs when the absolute estimate exceeds \(c\). As the estimate moves, it can leave and later re-enter the interval's exclusion region on the opposite side.
Equating a significance threshold with substantive importance. Whether an interval excludes zero and whether an effect exceeds a clinically meaningful threshold are distinct questions. State which one defines the tipping point.

Key takeaway: A delta analysis makes assumptions about missing outcomes explicit by shifting imputations in specified patterns. Its central result is conditional: it shows how the causal estimate and a clearly defined conclusion change under those shifts, not which shifts are true.

Check Your Understanding

Answer each question using the stated sign convention and distinguish sensitivity assumptions from observed-data information.

If imputed missing outcomes in one treatment arm are shifted upward by 2 points, what does that say relative to the reference imputation model, and which outcomes remain unchanged?
In a treatment-minus-control mean difference, why does a positive shift in missing control outcomes reduce the estimated effect?
Suppose 15% of one arm belongs to a missingness pattern and that pattern receives a delta of \(-4\) points. Under the weighted-mean setup, by how many points does that arm's mean change?
For a two-sided interval with fixed margin \(c\), state the condition for excluding zero. Why can exclusion return after being lost along a path where the estimate keeps decreasing?
What information does a tipping point provide, and what does it not establish about the unobserved outcomes?