Evaluating Sensitivity Curves for Pooled Estimates

Why Evaluate a Sensitivity Curve?

A delta adjustment represents a specified departure from the imputation model’s reference assumptions for missing values. When the adjustment varies over a range of plausible values, the resulting pooled estimates show how sensitive the substantive estimate is to that choice. A sensitivity curve makes this pattern visible: its direction indicates how the estimate changes as delta changes, while its steepness and curvature describe how that change behaves across the grid.

The curve is a descriptive summary of analyses under a sequence of assumptions. It does not establish which delta is correct, and it does not by itself summarize uncertainty in the estimates. Its usefulness depends on making the delta’s meaning clear, holding the rest of the analysis fixed, and plotting enough values to reveal the pattern without implying more precision than the grid supports.

Here, “pooled estimate” means the point estimate obtained by combining the analysis estimates from the imputations at that particular delta value. Pooling is performed separately at each grid value; estimates from different delta values are not pooled together. The curve’s vertical coordinate must refer to the same estimand, analysis model, and units at every point.

Conditions for a Meaningful Curve

A curve is interpretable only if its points differ in the intended way. The delta parameter should have a stated interpretation and scale, the grid should cover a range justified for the application, and all other choices should be held fixed unless the purpose is explicitly to vary them too. For example, changing both delta and the analysis model across points would make the curve unable to isolate sensitivity to delta.

The imputation and analysis procedures must also be applied consistently at each grid value. In practice, this means checking that the delta adjustment was implemented as intended, the chained equations have converged sufficiently for the analysis, and there are enough imputations for the pooled point estimates to be stable. These checks do not prove that the missing-data assumptions are correct. They help ensure that the plotted changes reflect the chosen delta scenarios rather than implementation errors or unstable computation.

The curve is a finite set of evaluated points. Connecting adjacent points with line segments is a visual aid, not proof that the estimate changes linearly between them. A smooth curve fitted through sparse points can suggest unsupported behavior, particularly near the ends of the grid. If the pattern between grid values matters, evaluate additional delta values there.

Direction, Slope, and Curvature

For two evaluated values \(d_a\) and \(d_b\), the secant slope is the change in pooled estimate divided by the change in delta. It describes the average rate of change over that interval, not necessarily the instantaneous rate at any point. Its units are the estimate’s units per unit of delta. If delta and the estimate share units, the slope is often dimensionless.

On an equally spaced grid with spacing \(h\), adjacent secant slopes can be compared to assess curvature. An increase in successive slopes indicates that the curve is bending upward over those grid intervals; a decrease indicates downward bending. The finite second difference at an interior grid point is

Dividing this second difference by \(h^2\) gives a finite-difference approximation to a second derivative when the curve is sufficiently smooth. Without that smoothness assumption, the second difference is still a useful description of how neighboring secant slopes change, but it should not be presented as an exact derivative.

Practical Stability Is a Substantive Judgment

A curve can change without changing enough to matter for a particular application. To describe practical stability, specify a tolerance in the units of the estimand and a reference estimate, such as the estimate at \(d=0\). The tolerance should be justified by clinical, policy, or scientific considerations before interpreting the plotted pattern. It is not a universal statistical cutoff.

For example, one may call estimates practically stable at evaluated grid points when their absolute difference from the reference estimate does not exceed a prespecified tolerance. This statement concerns the point estimates and the chosen tolerance; it is not a claim that the estimates are statistically indistinguishable. The curve alone cannot answer that inferential question.

Reading the Plot and Checking Its Construction

Plot delta on the horizontal axis and the pooled estimate on the vertical axis. Label both axes with units, identify the reference value if one is used, and show the evaluated points. A line joining the points can help the reader follow the sequence, but the points should remain visible so the grid is not mistaken for a continuous, exactly known function. If comparing prespecified scenarios, use clear labels and ensure each curve represents the same estimand and analysis.

A curve that rises or falls steadily indicates directional sensitivity over the evaluated range, but the direction need not be assumed in advance. The impact of a delta adjustment on an estimand depends on which values are missing, how the adjustment is applied, and how the estimand is defined. Curvature can signal that the effect of changing delta varies across the range. It may also arise from nonlinear analysis or from Monte Carlo variation, so visual shape should be checked rather than interpreted automatically as a substantive feature.

• Confirm that the plotted delta values match the values used in the imputation procedure.
• Check that every point uses the same analysis population, estimand, model, and units.
• Inspect the pattern of point estimates as well as the connecting line; do not infer behavior beyond the evaluated range.
• If small irregularities are important to the interpretation, assess whether they persist with more imputations or repeated runs.
• Document the delta range and its rationale so that the curve’s scope is clear.

Common Pitfalls and Limitations

A frequent mistake is to read a steep slope as proof that the missing-data assumptions are implausible. A steep slope shows that the pooled estimate responds strongly to the chosen delta over that interval; plausibility must be judged using the meaning of delta and relevant subject-matter information. Conversely, a nearly flat curve over a narrow range does not show robustness to values outside that range.

Another mistake is to treat a point-estimate curve as a complete uncertainty analysis. It contains no interval information by itself and cannot establish whether a change is distinguishable from sampling or imputation uncertainty. It also cannot identify a uniquely correct delta. A curve complements, rather than replaces, the substantive justification for the sensitivity range and the appropriate inferential summaries.

A table of estimates is a reasonable alternative when only a few delta values are evaluated and exact values matter most. A plot is usually more informative for seeing direction, curvature, and gaps in the grid. Neither format compensates for a poorly chosen range, inconsistent analyses, or unstable imputations. Where several sensitivity parameters vary together, a one-dimensional curve cannot show their joint behavior; the interpretation must remain limited to the single parameter varied.