Two Different Routes from a Delta to an Estimate
A delta adjustment is a sensitivity assumption about the distribution of values that were not observed. But “missing values” can refer to different variables in an analysis. Shifting an unobserved outcome changes the outcome values entering the analysis directly. Shifting an unobserved covariate changes the covariate distribution used to adjust or standardize the outcome model. Those routes need not have the same effect on an estimate, even if the numerical delta is the same.
Let \(Y\) be an outcome, \(X\) a covariate, and \(A\) a treatment indicator. Let \(R_Y=1\) indicate that \(Y\) is observed and \(R_X=1\) indicate that \(X\) is observed. An MAR reference imputation specifies a conditional distribution for missing values using the variables in the imputation model. A delta adjustment modifies that reference distribution for the variable that is missing. The analysis is then repeated using the adjusted imputations.
The delta is defined on the scale of the variable being shifted. For a continuous outcome, an additive shift might add \(\delta_Y\) outcome units to an MAR-imputed value. For a continuous covariate, an additive shift might add \(\delta_X\) covariate units to an MAR-imputed value. These deltas are not interchangeable: they refer to different variables, scales, and conditional distributions.
Assumptions Before Comparing the Shifts
The comparison below uses a single incomplete variable at a time. It assumes that the MAR reference imputation model is adequately specified for the relevant conditional distribution, that the analysis model or standardization procedure is appropriate for the estimand, and that imputed values remain within a defensible support. The delta specifies a departure from the MAR reference; the observed data alone generally cannot determine its value.
In multiple imputation, each delta value defines a separate sensitivity analysis. For each value, adjust the appropriate missing-variable imputations, analyze each completed data set, and combine the estimates and variances using the usual multiple-imputation procedure. For nonlinear analyses, it is especially important to rerun the analysis after changing imputations rather than assuming that a simple arithmetic correction to a pooled estimate is valid.
Worked Example: Shifting Missing Outcomes in a Treatment Contrast
Suppose a hypothetical trial has 50 participants in each arm. The outcome is continuous, and 10 treatment-arm outcomes and 5 control-arm outcomes are missing. Under the MAR reference imputations, the mean of the missing outcomes is 10 in the treatment arm and 8 in the control arm. The observed outcome means are 12 in the treatment arm and 9 in the control arm. For this arithmetic illustration, assume the imputed means and observed means are treated as fixed.
The MAR completed-data treatment mean is \((40\times12+10\times10)/50=580/50=11.60\). The control mean is \((45\times9+5\times8)/50=445/50=8.900\). Thus the MAR-reference treatment-minus-control contrast is \(11.60-8.900=2.700\) outcome units.
Now suppose the missing treatment outcomes are shifted by \(\delta_T=-2\) units and missing control outcomes by \(\delta_C=-1\) unit. The completed-data means become \([40\times12+10\times(10-2)]/50=560/50=11.20\) and \([45\times9+5\times(8-1)]/50=440/50=8.800\). The adjusted contrast is \(11.20-8.800=2.400\) outcome units.
The change in the contrast is \((10/50)(-2)-(5/50)(-1)=-0.300\) outcome units. The result depends both on the size and direction of each shift and on the fraction missing in each arm. The delta changes the outcome values entering the contrast directly.
For a difference in means, the arithmetic behind this example can be written explicitly. If \(n_a\) is the total number in arm \(a\), \(m_a\) is the number with missing outcomes, and \(\delta_a\) is the additive shift for those outcomes, then the adjusted arm mean changes by \((m_a/n_a)\delta_a\). Therefore, the treatment-minus-control contrast changes by the treatment-arm change minus the control-arm change. This identity applies to this mean contrast with additive shifts; it is not a general shortcut for nonlinear estimands or regression coefficients.
Covariate Shifts Act Through the Analysis
When \(X\) is incomplete, shifting its imputations changes the covariate values used in the analysis. The effect on the estimate depends on how the estimand uses \(X\) and how the outcome model relates \(X\) to \(Y\). A covariate shift can alter an adjusted or standardized contrast, but it need not alter every analysis coefficient.
For example, consider a linear outcome model with treatment effect modification: \(E(Y\mid A,X)=\beta_0+\beta_A A+\beta_X X+\beta_{AX}AX\). The model-based treatment contrast at covariate value \(x\) is \(\beta_A+\beta_{AX}x\). If the target is the average of this contrast over the target population’s covariate distribution, it equals \(\beta_A+\beta_{AX}E(X)\). A shift in missing \(X\) can therefore affect the standardized contrast when \(\beta_{AX}\ne0\). This calculation assumes the specified linear model and a target distribution represented by the completed data; it is not a universal formula for covariate adjustment.
Worked Example: A Covariate Shift Changes a Standardized Contrast
Consider a hypothetical population of 100 people, with a continuous covariate \(X\) missing for 30. Under the MAR reference imputations, the mean \(X\) across all 100 completed records is 2.0. Suppose the fitted outcome model is \(E(Y\mid A,X)=5+2A+0.5X+0.4AX\), and the target is the model-standardized treatment contrast over this population.
At covariate value \(x\), the treatment prediction is \(5+2+0.5x+0.4x\), while the control prediction is \(5+0.5x\). Their difference is \(2+0.4x\). Under the MAR reference, the standardized contrast is \(2+0.4(2.0)=2.800\) outcome units.
Now shift each missing \(X\) upward by \(\delta_X=1\) covariate unit, leaving observed \(X\) unchanged. The overall mean \(X\) increases by \((30/100)(1)=0.300\), from 2.0 to 2.3. The adjusted standardized contrast is \(2+0.4(2.3)=2.920\) outcome units, an increase of \(0.120\). Equivalently, the change is \(0.4(0.300)=0.120\).
This illustration holds the model coefficients fixed to isolate the standardization pathway. In an actual analysis, imputed covariates can also change the fitted coefficients, so the model must be refitted in each completed data set. The fixed-coefficient arithmetic is not a substitute for that refitting.
Worked Example: A Covariate Shift Need Not Change a Conditional Treatment Coefficient
Suppose instead that the hypothetical outcome model is \(E(Y\mid A,X)=5+2A+0.5X\), with no treatment-by-covariate interaction. The conditional treatment contrast at every \(x\) is \((5+2+0.5x)-(5+0.5x)=2\) outcome units. Standardizing this contrast over a covariate distribution with mean 2.0 gives 2.0; standardizing over one with mean 2.3 also gives 2.0.
Thus, under this correctly specified linear model and this standardized contrast, the covariate shift changes predicted outcome levels but cancels from the treatment contrast. It does not follow that covariate deltas are generally irrelevant: they may alter a marginal mean, a nonlinear-model contrast, an interaction-based contrast, or fitted coefficients after refitting.
Why the Difference Matters
The two adjustment types answer different sensitivity questions. An outcome delta asks how conclusions change if the missing outcomes differ from their MAR-reference imputations, conditional on the imputation model. A covariate delta asks how conclusions change if the missing covariates differ from their reference imputations. The latter then propagates through the outcome analysis, possibly affecting both fitted relationships and the distribution over which predictions are averaged.
| Feature | Missing outcome shift | Missing covariate shift |
|---|---|---|
| Distribution modified | Conditional distribution of \(Y\) among records with missing \(Y\) | Conditional distribution of \(X\) among records with missing \(X\) |
| Immediate route to the estimate | Changes outcomes entering the analysis | Changes covariate values used for adjustment, prediction, or standardization |
| Scale for an additive delta | Outcome units | Covariate units |
| Effect on an estimate | Often direct for means; depends on the estimand and missingness pattern | Depends on the outcome model, estimand, and refitting procedure |
A covariate adjustment also requires care about the imputation model. If \(Y\) is observed while \(X\) is missing, the reference imputation for \(X\) may condition on \(Y\), along with other relevant predictors. A delta shift then describes departure from that specified conditional reference distribution; it does not justify omitting important predictors or treating imputed covariates as known.
Diagnostics and Common Pitfalls
For each delta value, inspect whether adjusted imputations are plausible for the variable’s scale and support. Compare distributions of observed and imputed values, by relevant groups, and track the analysis estimate as the delta changes. Check for implausible predicted outcomes or covariates, extreme extrapolation, and convergence or instability in the imputation and analysis models. Large shifts can expose model limitations rather than provide a credible description of missing values.
- Wrong: “A delta of 1 means the same sensitivity change whether I shift the outcome or the covariate.” Correction: A delta is measured on the scale of the variable being shifted, and its impact depends on how that variable enters the estimand.
- Wrong: “A covariate shift must change the treatment effect.” Correction: A shift may cancel from a particular contrast, as in a correctly specified linear model with a constant treatment effect. Other estimands or models can behave differently.
- Wrong: “The covariate-shift effect can always be calculated by changing the covariate mean in the pooled estimate.” Correction: That shortcut requires restrictive model and estimand conditions. In general, adjust imputations, refit the analysis, and recompute the estimand in every completed data set.
- Wrong: “The delta is learned from observed data.” Correction: A delta encodes a sensitivity assumption about values not identified by the observed data alone. Its range needs substantive justification.
The key practical distinction is whether the changed values enter the estimand directly as outcomes or alter the covariates on which the outcome analysis depends. A sound sensitivity analysis states the variable shifted, the conditional reference distribution, the delta scale, and the target estimand before interpreting how the estimate changes.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In a treatment arm of 80 people, 20 outcomes are missing. An additive outcome delta is \(-1.5\) units. By how much does that arm’s completed-data mean change?
- A continuous covariate has 25% missing values. An additive covariate delta is \(+2\) units. What is the change in the completed-data covariate mean if the MAR reference imputations and observed values are held fixed?
- For the model \(E(Y\mid A,X)=\beta_0+\beta_AA+\beta_XX\), explain why shifting \(X\) need not change a standardized treatment contrast.
- For \(E(Y\mid A,X)=3+1.5A+0.2X+0.5AX\), the target covariate mean rises by 0.4 after a delta adjustment. What is the change in the model-standardized treatment contrast, holding coefficients fixed?
Solutions. (1) The changed fraction is \(20/80=0.25\), so the mean changes by \(0.25(-1.5)=-0.375\) outcome units. (2) The mean changes by \(0.25(2)=0.500\) covariate units. (3) The treatment contrast at any \(x\) is \(\beta_A\), since the \(\beta_Xx\) term cancels between treatment and control; averaging that constant over a changed \(X\) distribution still gives \(\beta_A\). (4) The interaction coefficient is 0.5, so the contrast changes by \(0.5(0.4)=0.200\) outcome units under the stated fixed-coefficient linear standardization calculation.