Why an Extreme Delta Needs Interpretation
A delta-adjusted imputation can show how an analysis changes when the missing outcomes are assumed to differ from their predictions under a reference model. But a large change in the final estimate does not, by itself, make the chosen delta informative. The first question is what that delta implies for the outcomes of people whose values are missing. A delta may describe a severe but interpretable stress test, or it may imply values that conflict with the outcome’s scale, clinical meaning, or known constraints.
The assessment is conditional on the reference imputation model and the chosen delta parameterization. It does not establish what the unobserved outcomes actually were. In particular, the observed outcomes cannot by themselves verify a missing-not-at-random assumption. The practical goal is to make the sensitivity assumption transparent enough to judge and, where possible, to motivate its range using subject-matter knowledge.
For example, suppose a reference model predicts a mean outcome of 60 for a particular covariate pattern. An additive \(\delta=-10\) shifts the implied mean to 50 for missing cases at that pattern. This is not a claim that every missing value is exactly 10 units lower. It describes a shift in the conditional distribution; individual imputed values still vary according to the model. For a binary outcome, a delta of \(-1\) does not mean a one-percentage-point decrease. It multiplies the event odds by \(e^{-1}\), and the probability change depends on the reference probability.
Translate Delta into an Implied Distribution
Before judging an extreme value, state the reference model, the scale on which delta operates, and the group or covariate patterns to which it applies. For an additive shift, if the reference predictive distribution for a missing outcome at covariates \(X=x\) has mean \(\mu_0(x)\) and standard deviation \(\sigma(x)\), the shifted distribution has mean \(\mu_0(x)+\delta\), with the same spread when the adjustment is a pure location shift. The standardized shift \(\delta/\sigma(x)\) helps express its size relative to predictive variability, but it does not replace checking the outcome’s actual units and limits.
For a binary event, let \(p_0(x)\) denote the reference event probability. Under a logistic delta adjustment, the shifted probability is
This follows because \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). Adding \(\delta\) to the log odds multiplies the odds \(p/(1-p)\) by \(\exp(\delta)\); converting those shifted odds back to a probability gives the expression above. Thus, the same delta generally produces different absolute probability changes at different reference probabilities.
For an extreme scenario, assess the full implied distribution rather than only its mean or the eventual analysis estimate. Useful checks include the fraction of predicted values outside legal bounds, the implied probability of clinically important states, and whether the distribution differs sensibly across covariate patterns. A valid software run or a converged imputation chain does not establish that the resulting assumptions are plausible.
Worked Example: A Severe Shift for a Bounded Score
Suppose a hypothetical symptom-improvement score ranges from 0 to 100, with higher values indicating greater improvement. For a covariate group with missing scores, the reference predictive distribution is approximated by a normal distribution with mean 62 points and standard deviation 12 points. Consider additive deltas of \(-30\) and \(-50\) points. For this diagnostic calculation, assume the shifted reference distribution retains the standard deviation of 12 points.
With \(\delta=-30\), the implied mean is \(62-30=32\) points. Under the stated normal approximation, the probability of an impossible score below zero is
That is about 0.383% below the lower bound. The upper-bound probability is \(P(Y>100)=1-\Phi((100-32)/12)=1-\Phi(5.667)\), which is negligible under this approximation. This shift is severe relative to the reference mean, but the calculation does not reveal a large fraction of impossible draws. Its clinical plausibility still requires considering whether an average of 32 points is credible for the missing group.
With \(\delta=-50\), the implied mean is \(62-50=12\) points. The lower-bound probability becomes
About 15.87% of the implied normal draws are below the score’s minimum. That is a substantial support violation. The \(-50\) scenario may still be deliberately retained as a mathematical stress test, but the normal shifted model should not be described as a realistic distribution on a 0-to-100 scale. A bounded or otherwise appropriate predictive model, with its own clearly specified sensitivity adjustment, may be needed to represent such a scenario coherently.
Check Subject-Matter Constraints, Not Just the Scale
A value can fall within the numerical range and still be unreasonable. Consider treatment adherence, recovery time, or a count of serious events: bounds may rule out some values, while clinical knowledge may make other values implausible. Assessment should therefore ask both whether the implied distribution respects formal constraints and whether its location, spread, and important tail probabilities make sense in context.
The relevant comparison is usually conditional. A single overall mean can conceal extreme assumptions for a subgroup or covariate pattern. If the reference model gives different predictions across \(x\), examine \(\mu_0(x)+\delta\) or \(p_\delta(x)\) for representative values of \(x\). Also check whether a constant delta is being applied on a scale where a constant shift is meaningful. A fixed additive shift in a continuous outcome has outcome units; the same numerical shift can be mild for a high-variability outcome and severe for a low-variability one.
Worked Example: Interpreting an Extreme Binary-Outcome Delta
In a hypothetical study, the reference imputation model gives an event probability of \(p_0=0.20\) for a covariate pattern among people with missing outcomes. The event is a clinically important adverse outcome. Apply a logistic delta of \(-3\), which lowers the event odds for the missing group relative to the reference model.
The reference odds are \(0.20/(1-0.20)=0.25\). The odds multiplier is \(\exp(-3)\approx0.04979\), so the shifted odds are
Convert these odds to a probability:
Thus, the implied event probability is about 1.229%, rather than 20%. The delta has multiplied the event odds by about 0.04979, a reduction of roughly 95.02% in the odds. It has not subtracted three percentage points. Whether a reduction from 20% to about 1.23% is a meaningful stress test depends on the event, the covariate group, and clinical knowledge. If such a low event rate is incompatible with external evidence or known mechanisms, the scenario may be useful to show how sensitive the conclusion is, but it should not be presented as a credible central assumption without justification.
A key distinction is between extreme and implausible. An extreme scenario can be intentional: it may ask how far the missing outcomes would have to depart from the reference model before the substantive conclusion changes. Its value is then as a stress test, not as a best estimate of the missing-data mechanism. Conversely, an apparently modest delta may imply an implausible distribution when the reference predictions are already near a boundary or when the relevant subgroup has little variability.
Worked Example: A Clinically Impossible Tail for Recovery Time
Suppose a hypothetical recovery-time outcome is measured in days, is known to lie between 0 and 30 days, and has a reference predictive distribution for a missing subgroup approximated by a normal distribution with mean 8 days and standard deviation 6 days. Consider an additive \(\delta=25\) days. Under the pure location-shift assumption, the implied mean is \(8+25=33\) days.
The probability of an imputed value exceeding the known maximum is
So about 69.15% of the implied normal draws exceed 30 days, a known upper limit. This is not a subtle issue of interpreting a large mean: the assumed shifted normal distribution puts most of its mass outside the outcome’s support. The calculation diagnoses a mismatch between the normal location-shift model and this extreme delta. If the aim is a bounded stress test, specify a distribution and adjustment that respect the 0-to-30-day range, and then assess the resulting implied distribution. Simply truncating draws after imputation would change the model and should not be treated as an innocuous repair.
A Practical Plausibility Assessment
A disciplined assessment makes the scenario’s meaning explicit before using it to interpret changes in an analysis. One useful sequence is:
Identify the outcome scale, the reference conditional predictions, and the assumed spread or link function. A delta has meaning only relative to this reference.
For a continuous outcome, calculate shifted means and relevant quantiles in outcome units. For a binary outcome, convert shifted log odds to probabilities for representative reference risks.
Calculate the mass outside known bounds and examine clinically important tails, subgroup patterns, and consistency with available subject-matter or external information.
Explain whether it is intended as a credible sensitivity range, a deliberately severe stress test, or an illustration of model failure. Do not confuse a stress test with an evidence-based prediction.
The main alternatives are not interchangeable. A reference-model scenario without a delta is a useful starting point but relies on its missingness assumptions. A calibrated delta range can be anchored to external data or elicited expert judgments, though those sources may themselves be uncertain. A worst-case or bounds-based analysis can expose what follows under broad constraints, but may be too conservative to represent a realistic mechanism. Alternative outcome models can respect bounded, discrete, or skewed outcomes more naturally, but their assumptions also require scrutiny. No one approach makes the unobserved outcomes identifiable from the observed data alone.
Diagnostics, Limitations, and Common Mistakes
Warning signs include a large proportion of imputed values beyond the outcome’s support, probabilities of events that conflict with established knowledge, sharp and unexplained changes across adjacent covariate patterns, or conclusions that depend on an adjustment whose units have been misunderstood. Inspecting only pooled estimates can miss these problems; examine the implied distributions for missing cases as well. In sparse groups, estimates of the reference distribution may be unstable, so apparently precise implied probabilities should not be overinterpreted.
- Mistake: calling an extreme delta “plausible” because the imputation software ran. Successful computation establishes neither scientific credibility nor compatibility with the outcome scale. Calculate and interpret the implied distribution.
- Mistake: treating an additive shift as a fixed change for every missing value. A location shift moves the distribution; individual values still vary. State whether the adjustment changes only location or also other distributional features.
- Mistake: interpreting a logistic delta as a probability difference. A logistic delta multiplies odds by \(\exp(\delta)\). Convert to probabilities at relevant reference risks before describing its size.
- Mistake: discarding every implausible scenario. A deliberately implausible stress test can be informative if clearly labelled. The error is presenting it as a credible assumption, not necessarily including it in a robustness analysis.
Plausibility judgments are necessarily conditional on context and often uncertain. They do not certify a delta as true, and they cannot prove that a less extreme scenario is correct. Their purpose is to prevent numerical sensitivity parameters from becoming detached from the outcomes they are meant to represent.
Check Your Understanding
Answer each question, then compare with the concise solutions below.
- A reference continuous-outcome prediction has mean 40 units. What mean does an additive delta of \(-12\) imply, and what additional information is needed to judge the shift’s severity?
- A binary reference event probability is 0.10. Under logistic delta \(\log(2)\), what is the shifted event probability?
- A normal imputation model for a score bounded between 0 and 50 has shifted mean 45 and standard deviation 10. What proportion is predicted above 50, and what does that suggest?
- Why can an extreme scenario be useful even when it is not considered clinically plausible?
Solutions. (1) The shifted mean is \(40-12=28\) units. Assessing severity also requires the predictive spread, outcome bounds, and subject-matter context; 12 units alone is not enough. (2) The reference odds are \(0.10/0.90=1/9\). Multiplying by \(\exp\{\log(2)\}=2\) gives odds \(2/9\), hence probability \((2/9)/(1+2/9)=2/11\approx0.1818\), or 18.18%. (3) \(P(Y>50)=1-\Phi((50-45)/10)=1-\Phi(0.5)\approx0.3085\), or 30.85%; substantial mass above the score maximum indicates a support problem for this shifted normal model. (4) It can show how robust the analysis is under a deliberately severe departure from the reference assumption, provided it is labelled as a stress test rather than a credible prediction.