Why Inspect the Implied Distributions?
A delta value encodes a departure from the assumptions used to generate imputations under a reference model. A useful sensitivity analysis therefore asks more than whether the delta-adjusted analysis changes a final estimate: it asks what outcome values the assumption implies for cases with missing data. Quantiles, event rates, and plots make those implications visible on the outcome’s actual scale.
The comparison is a diagnostic, not a test of whether missing values have been recovered correctly. Observed outcomes belong to cases with observed data; imputed outcomes belong to cases with missing data. Those groups can differ in their observed covariates and missingness patterns. Their marginal distributions need not match even when an imputation model is appropriate. Compare like with like where possible, and interpret every discrepancy in light of who is represented in each group.
Set Up a Fair Comparison
Let \(Y\) denote the incomplete outcome and \(R=1\) indicate that \(Y\) is observed; \(R=0\) indicates that it is missing. For each completed data set and each specified delta value, collect the imputed values for cases with \(R=0\). Compare these with the observed values for \(R=1\), preferably within comparable covariate groups or missingness patterns when that is relevant and the data permit it.
Before inspecting results, specify the delta grid, the outcome scale, any clinically or scientifically meaningful event threshold, and the summaries to report. A grid should reflect plausible assumptions informed by subject-matter judgment or external information, rather than being selected only after seeing which values produce a preferred conclusion. The earlier material on standardizing delta parameters across outcomes is relevant when the same numerical shift would have different meanings on different scales.
For a continuous outcome using an additive shift, the adjusted imputation is \(Y_{\delta}=Y_0+\delta\), where \(Y_0\) is the imputation under the reference scenario. For a binary outcome, an odds-scale shift adds delta to the log odds, so the adjusted event probability is \(\operatorname{expit}\{\operatorname{logit}(p_0)+\delta\}\). These interpretations depend on applying the specified adjustment to the intended missing cases and on any range constraints being handled as specified. A delta is not automatically measured in outcome units: its meaning follows the scale on which it is defined.
What the Summaries Show
Report several quantiles, such as the 10th, 25th, 50th, 75th, and 90th percentiles, rather than relying on the mean or median alone. Quantiles can reveal whether a shift changes the center, spreads the tails, or produces an unusual concentration. For a binary outcome, report the imputed event rate as well as the probability distribution or the range of imputed probabilities across cases. For a continuous outcome, an event rate can also be informative if the threshold is meaningful and fixed in advance—for example, the fraction above a prespecified clinical cut-point.
Plots add information that a short table cannot. Histograms, density plots, or empirical distribution functions can reveal multimodality, gaps, skewness, and tail behavior. Overlaying the observed and imputed groups is useful, but label the groups clearly and avoid implying that they are exchangeable. When there are multiple completed data sets, show whether the distributional pattern is stable across imputations; do not treat all imputed values as independent observations for an ordinary inferential test.
Worked Example: Additive Shifts and Quantiles
Suppose a hypothetical data set has two observed outcome values, 6 and 8 units, and two missing outcomes. Under the reference imputation model, the two missing cases receive values 3 and 5 units. Consider additive delta values of 0, 2, and 4 units. For this illustration, assume the adjustment is applied directly, without a range constraint.
At \(\delta=0\), the imputed median is \((3+5)/2=4\) units. At \(\delta=2\), the adjusted values are 5 and 7, giving median \((5+7)/2=6\) units. At \(\delta=4\), the values are 7 and 9, giving median \((7+9)/2=8\) units. The observed median is \((6+8)/2=7\) units. Thus the imputed median moves from 4 to 6 to 8 units across this grid; the observed median of 7 is a descriptive reference, not a target that the imputed median must equal.
For a threshold of 6.5 units, the observed event rate is \(1/2=0.50\), since only 8 exceeds the threshold. Among the two imputed values, the event rate is 0/2 at \(\delta=0\), 1/2 at \(\delta=2\), and 2/2 at \(\delta=4\). These calculations show that the grid changes both the imputed center and the proportion above the threshold. With only two values per group, these rates are highly coarse; in a real analysis they would be summarized over the relevant missing cases and checked across completed data sets.
Why Additive Shifts Move Quantiles
For an unrestricted additive adjustment, the quantile movement follows directly from the definition of a quantile. If \(F_0(y)\) is the distribution function of \(Y_0\), then \(Y_{\delta}=Y_0+\delta\) has distribution function
Consequently, for a consistently defined \(p\)th quantile, \(Q_p(\delta)=Q_p(0)+\delta\). This identity is a useful check on an implementation of a pure additive shift. It need not hold after truncation, rounding, or another constraint changes values, and it does not mean the observed and imputed quantiles should move together: only the adjusted imputations are shifted.
Worked Example: An Odds-Scale Shift for a Binary Outcome
Consider ten hypothetical missing cases: five have reference event probability \(p_0=0.200\), and five have \(p_0=0.600\). Apply an odds-scale delta of \(0.200\). For a reference probability \(p_0\), the adjusted probability is
For the first five cases, \(p_{\delta}=0.200e^{0.200}/(0.800+0.200e^{0.200})\approx0.2339\). For the other five, \(p_{\delta}=0.600e^{0.200}/(0.400+0.600e^{0.200})\approx0.6469\). The average adjusted event probability across these ten cases is \((5(0.2339)+5(0.6469))/10\approx0.4404\). The reference average is \((5(0.200)+5(0.600))/10=0.4000\).
The implied event rate has increased by about 0.0404, or 4.04 percentage points, in this hypothetical set of missing cases. The response is not a uniform increase of 0.200 in probability: delta shifts log odds, and the probability change depends on the starting probability. If the observed event rate were 0.450 in a comparable observed group, that would provide context for judging the implied rate, but would not establish that delta \(=0.200\) is correct.
Plot and Diagnose the Grid
A compact diagnostic display can place delta values in rows and distribution summaries in columns. For a continuous outcome, include selected quantiles and any prespecified threshold rate. For a binary outcome, include event rates and, when useful, the distribution of case-specific imputed probabilities. Add plots that show observed values and imputed values separately, with clear labels and consistent axes across delta scenarios. Keeping the same axes prevents visual rescaling from disguising the size of a shift.
Look for patterns that conflict with the outcome’s substantive scale or known features. A scenario may imply implausible tail values, a sudden pile-up at a boundary, or an event rate outside a credible range. Also inspect whether a supposed shift has little or no effect because constraints repeatedly clip values, or whether results vary greatly across completed data sets. Such patterns call for checking the adjustment implementation, the scale of delta, the handling of constraints, and the relevance of the chosen subgroup comparisons. Boundary behavior from large shifts requires more focused diagnosis and is treated separately.
If imputations are generated under several delta values, keep the scenario-specific summaries distinct. Combining all delta scenarios into one distribution hides the assumptions being compared. Likewise, a smooth-looking plot is not evidence of a correctly specified missing-data model. A plot can expose a contradiction or coding problem; it cannot identify the unobserved distribution from observed data alone.
Worked Example: Reading a Quantile and Event-Rate Display
Suppose a hypothetical clinical outcome is measured in points, and 40 cases have missing outcomes. A diagnostic summary gives the following imputed quantiles and the proportion above a fixed threshold of 10 points. Observed outcomes in a covariate-comparable group have median 8 points and an observed proportion above 10 of 0.30.
| Delta (points) | Imputed 10th percentile | Imputed median | Imputed 90th percentile | Imputed proportion above 10 |
|---|---|---|---|---|
| −2 | 1 | 5 | 11 | 0.20 |
| 0 | 3 | 7 | 13 | 0.35 |
| 2 | 5 | 9 | 15 | 0.55 |
The table shows an orderly movement: each reported quantile rises by 2 points when delta rises by 2 points, consistent with an unrestricted additive shift. The imputed median at delta 0 is 7 points, compared with 8 points in the observed comparison group; the proportions above 10 are 0.35 and 0.30, respectively. At delta 2, the imputed median is 9 and the event proportion is 0.55. This scenario implies a noticeably larger upper-tail proportion than the observed reference, which merits subject-matter review. It is not automatically implausible, since the missing cases may differ from the observed group. A plot would help determine whether the difference reflects a broad shift or an unusual tail, while subgroup information would help assess whether the comparison is fair.
Assumptions, Alternatives, and Common Pitfalls
These checks assume that the imputed values were generated using the stated delta convention, that the summaries refer to the intended missing cases, and that the outcome and threshold are measured consistently. A marginal observed-versus-imputed comparison is most interpretable when the groups are reasonably comparable; otherwise, stratification or covariate-standardized summaries may be more informative. Even then, the diagnostics depend on the chosen model and cannot verify the distribution of values that were never observed.
A single reference scenario is simpler to present but does not show how assumptions alter the implied distribution. Numerical summaries are compact and easy to compare across a grid, while plots reveal shape and unusual concentrations; using both is generally more informative than choosing one. External information can help determine whether a scenario is substantively credible, but it should be used to justify the delta range rather than to claim that observed and missing groups must have identical marginal distributions.
- Pitfall: matching the observed median by construction. The reasoning “the observed median is 7, so the imputed median must also be 7” ignores possible differences between observed and missing cases. Use the comparison as context, and justify the scenario using the missingness structure and substantive information.
- Pitfall: interpreting delta as a probability-point change in a binary model. An odds-scale delta changes log odds, not probability by a constant amount. Apply the expit formula to each reference probability before summarizing event rates.
- Pitfall: treating a distribution plot as a validation test. A visually similar pair of distributions does not prove the missing-data assumptions, and a difference does not by itself prove an imputation failure. Check comparability, model specification, and substantive plausibility.
- Pitfall: overlooking constraints or scale errors. If a pure additive shift should move each quantile by delta but does not, check whether truncation, rounding, or implementation details changed the adjustment.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- Under an unrestricted additive shift of 1.5 units, a reference median is 4 units. What is the adjusted median, and what assumption is needed for this calculation?
- A binary outcome has reference probability 0.25 and an odds-scale delta of 0.4. Should the adjusted probability be reported as 0.65? Explain.
- The observed event rate is 0.40 and the imputed event rate is 0.60. Does this difference prove that the delta is too large? Why or why not?
- In an additive-shift analysis, the reported quantiles barely change as delta increases. Name two features that could explain this pattern and one implementation check.
Solutions. (1) The adjusted median is \(4+1.5=5.5\) units, assuming the shift is applied without constraints or other transformations. (2) No. Delta adds 0.4 to log odds; the probability is \(0.25e^{0.4}/(0.75+0.25e^{0.4})\approx0.3321\), not 0.65. (3) No. The groups may differ, and the comparison is diagnostic rather than a test of the unobserved outcomes; assess comparability and substantive plausibility. (4) Constraints such as truncation may prevent values from moving, or the summaries may have been computed for the wrong cases or scale. Check that the adjustment is applied to the intended missing cases and that the delta uses the specified outcome scale.