Diagnosing Implausible Delta-Adjusted Predictions

Why Inspect Adjusted Predictions?

A delta adjustment expresses a sensitivity assumption about outcomes that are missing. Even when a chosen delta is defensible, its consequences may not be. Predictions can fall outside the outcome’s possible range, change sharply across a subgroup boundary, or imply a longitudinal course that is difficult to reconcile with the clinical setting. These are reasons to investigate the model and the assumptions—not automatic proof that a particular delta is invalid.

The earlier material on checking the reference model provides the starting point: assume the reference prediction model has been examined for its intended use. The task here is different. We inspect what happens after applying the adjustment, including how that result behaves across outcome values, subgroups, and time. The observed data do not reveal the missing outcomes, so these checks assess implications and coherence rather than determine the true delta.

This definition describes a distributional shift, not merely an instruction to add delta to a reported average. A check based only on adjusted means can miss implausible tails or individual trajectories. Conversely, a randomly drawn imputed value outside a plausible central range is not automatically impossible: it matters whether it lies outside the outcome’s actual support, and how much predictive probability is assigned there.

Start With Support and Scale

Before plotting, state what values the outcome can take and what its units mean. A score may be bounded by a test scale, a count cannot be negative, and a proportion lies between 0 and 1. Some limits are logical; others are clinical or operational and require justification. Keep these categories distinct. A value can be rare or clinically concerning without being mathematically impossible.

For a continuous outcome with support \([L,U]\), an additive delta shifts the reference distribution. The adjusted probability below or above the support is therefore

These expressions make the diagnostic concrete: evaluate the reference distribution at the shifted boundaries. For discrete outcomes, use the corresponding probabilities at values outside the allowed set, taking account of whether the boundary values themselves are allowed. If a predictive model already respects the outcome’s support, a location shift may not. A large out-of-support probability is a warning about combining that reference distribution with that additive shift.

Look for Abrupt Subgroup Changes

When the adjustment depends on treatment arm or a clinically distinct subgroup, compare the reference and adjusted predictions side by side. Useful summaries include subgroup means, selected quantiles, predicted probabilities above a clinical threshold, and contrasts between subgroups. Plotting these summaries often makes a discontinuity easier to see than reading a table of delta parameters.

A discontinuity is not necessarily a flaw. If the delta is deliberately different between groups, the adjusted predictions may also differ. The diagnostic question is whether the size and direction of that change have a credible interpretation given the scientific rationale for the subgroup-specific assumption. Check that the change is not driven by a coding error, a sparse subgroup, or an unintended interaction.

Inspect Trajectories, Not Only Time-Specific Means

For repeated outcomes, a collection of individually plausible time-specific predictions can still form an implausible path. Plot adjusted trajectories against time for representative histories, and compare their increments with the reference trajectories. Examine whether shifts create sudden jumps, reversals, or sustained rates of change that conflict with the outcome’s context. Do not assume a smooth trajectory is always correct; some outcomes change abruptly. The relevant benchmark is clinical and design knowledge, not smoothness for its own sake.

For equally spaced times, a simple numerical check is to calculate adjacent increments. If adjusted means at times \(t_1,\ldots,t_k\) are \(m_1,\ldots,m_k\), the increment from \(t_j\) to \(t_{j+1}\) is \(m_{j+1}-m_j\). Large or alternating increments can flag a pattern for review. With unequal time intervals, compare rates of change by dividing each increment by the elapsed time.

A Practical Diagnostic Sequence

A disciplined review combines simple numerical summaries with plots. Use the same prediction records for reference and adjusted summaries so that differences reflect the adjustment rather than a changed target population.

Numerical checks should be chosen to match the concern. For a bounded outcome, report out-of-support probability. For a subgroup adjustment, report contrasts before and after adjustment. For longitudinal predictions, report adjacent increments and, when useful, second differences. No single number substitutes for inspecting the distribution and the context.

What These Checks Can and Cannot Establish

An implausible adjusted prediction can arise from several sources: an overly strong delta, an unsuitable reference distribution, a support mismatch, an indexing or coding mistake, or a real feature of the scientific scenario that needs clearer explanation. The diagnostic alone does not distinguish these causes. In particular, observed outcomes cannot verify the assumed distribution of outcomes among those missing. A plot that looks reasonable is not evidence that the unobserved outcomes follow the adjusted distribution.

The main alternatives answer different questions. Reference-model diagnostics assess the reference predictions before adjustment. Delta sensitivity analysis compares predictions under alternative assumptions about missing outcomes. Distributional approaches, such as a bounded-outcome model or an adjustment of a different distributional feature, may better respect the scale in some settings. None removes the need to state the assumption. Mechanically clipping values, smoothing subgroup changes, or deleting uncomfortable predictions can conceal rather than solve a mismatch.

Treat diagnostics as a reasoned review, not a significance test. There is generally no universal cutoff for an acceptable subgroup jump or trajectory curvature. Document the feature, its size and units, the groups or histories affected, and the scientific rationale for retaining or revising the specification. If a different adjustment is considered, recompute and inspect its implications rather than assuming it fixes the problem.