Calibrating Deltas to External Information

Why Calibrate a Delta?

A delta adjustment makes an explicit assumption about how the distribution of a missing outcome differs from the distribution predicted using observed outcomes and measured covariates. Subject-matter judgment can identify a plausible direction and range for that shift. External studies and validation samples can add quantitative anchors: they may suggest how large a shift is credible, provided their information is relevant to the population and outcome being imputed.

Calibration is not simply copying a published number into an imputation model. The external contrast must refer to the relevant groups, align with the model’s conditioning variables, and be translated to the model’s outcome scale. A difference of six score points, a change in odds, and a rate ratio are not interchangeable quantities. Their numerical values cannot be compared without defining the scale and the contrast.

Definition: Let \(R=1\) denote an observed outcome and \(R=0\) a missing outcome, and let \(X\) denote the covariates used to predict the outcome. On a specified scale \(g\), a delta adjustment represents the assumed conditional shift \(g\{E(Y\mid R=0,X)\}=g\{E(Y\mid R=1,X)\}+\delta(X)\). For a continuous outcome modeled on its original scale, \(g\) is the identity function. For a binary outcome modeled with a logit link, \(g(p)=\log\{p/(1-p)\}\). For a count rate modeled with a log link, \(g\) is the logarithm of the rate. The sign and units of \(\delta\) are therefore defined by the chosen scale.

This equation describes the assumed relationship after conditioning on \(X\); it does not assert that the missing and observed groups have the same covariate distribution. In practice, a delta may be constant within a chosen group, or it may depend on covariates. A single external marginal difference is not automatically an estimate of a conditional delta: group composition can create or obscure a difference. The calibration target should match the conditional contrast relevant to the imputation model, or be adjusted to do so.

Translate the Evidence Before Using It

Begin by identifying exactly what the external information estimates. Is it a mean difference, a probability difference, a log-odds contrast, or a rate ratio? Then identify the model scale for the imputed outcome. If the scales differ, translate the external result using a justified conversion. A conversion is defensible only when the relationship between the external measure and the imputation outcome is known or supported; similar labels for two measurements do not establish equivalence.

When a validation sample records outcomes for people who would otherwise have missing outcomes, it can be especially informative. Its value depends on how those people were sampled and whether their outcomes are measured comparably. A study of a different population may still help, but transport requires stronger assumptions, such as adequate adjustment for relevant differences in covariates and setting.

Worked Example: Converting an External Score Difference

Suppose a hypothetical external validation study reports that a relevant group with missing follow-up outcomes scores, on average, 6 points below a comparable observed-outcome group on a 0-to-100 questionnaire. The MICE model uses the same construct on a 0-to-10 scale, with 10 external-scale points corresponding to 1 point on the model scale. The groups are assumed comparable after adjustment for the covariates in \(X\), and higher scores have the same meaning on both scales.

On the external scale, the estimated difference is \(-6\) points. The model-scale conversion is to divide by 10:

\delta = \frac{-6\ \text{external-scale points}}{10\ \text{external-scale points per model-scale point}} =-0.600\ \text{model-scale points}.

The calibrated central value is therefore \(\delta=-0.600\) points on the 0-to-10 scale. For example, if the observed-outcome model predicts a conditional mean of 7.2 points for a covariate profile, the shifted mean is \(7.2-0.600=6.60\) points. This calculation assumes the scale conversion applies to the relevant range and that the external contrast represents the conditional contrast of interest. If the study reports a confidence interval from \(-10\) to \(-2\) external points, dividing both endpoints by 10 gives a corresponding interval from \(-1.00\) to \(-0.200\) model-scale points. That interval reflects sampling uncertainty in the external estimate; it does not, by itself, capture uncertainty about transportability or the scale conversion.

For binary outcomes, an external probability difference should not be inserted directly as a logit delta. The appropriate translation is to calculate the log odds in each group and subtract. This distinction matters because an equal change in probability does not represent an equal change in log odds at different baseline probabilities.

Worked Example: Calibrating a Binary-Outcome Delta

In a hypothetical validation sample with covariates comparable to the target imputation population, 18 of 100 people with previously missing outcomes have the event. In a comparable observed-outcome group, 30 of 100 have the event. Treat these sample proportions as estimates of the corresponding conditional probabilities for this calculation.

The estimated probabilities are \(p_0=18/100=0.180\) for the missing-outcome group and \(p_1=30/100=0.300\) for the observed-outcome group. Under a logit-scale delta, the external contrast is:

\delta =\log\left(\frac{p_0}{1-p_0}\right) -\log\left(\frac{p_1}{1-p_1}\right) =\log\left(\frac{0.180}{0.820}\right) -\log\left(\frac{0.300}{0.700}\right) =\log\left(\frac{0.180(0.700)}{0.820(0.300)}\right) \approx -0.669.

The calibrated delta is approximately \(-0.669\) on the log-odds scale. To check its meaning, the observed-group odds are \(0.300/0.700=0.4286\). Multiplying by \(\exp(-0.669)\), approximately \(0.512\), gives odds of \(0.2195\). Converting those odds back to a probability gives \(0.2195/(1+0.2195)=0.180\), matching the validation proportion. The adjustment is thus an odds multiplier of about 0.512, not a decrease of 0.669 in probability. This calibration is only credible if the groups are comparable on relevant predictors and the validation proportions are informative about the target population.

For count outcomes, distinguish a change in the count from a change in a rate. If people contribute different amounts of exposure, the external contrast should generally be expressed in rates, with the exposure definition aligned across groups. A log-rate delta is the logarithm of the rate ratio, not the raw difference in event counts.

Worked Example: Calibrating a Count-Rate Delta

Consider a hypothetical validation study. Among people representing the missing-outcome group, 72 events occur over 40 person-years. Among a comparable observed-outcome group, 96 events occur over 40 person-years. Assume the event definitions and exposure measurement are the same in both groups and that the rates are appropriate summaries for the imputation model.

The estimated rates are \(72/40=1.80\) and \(96/40=2.40\) events per person-year. Their ratio and the log-rate delta are:

\text{Rate ratio}=\frac{1.80}{2.40}=0.750, \qquad \delta=\log(0.750)\approx -0.288.

If the observed-outcome model predicts 2.40 events per person-year for a comparable profile, applying this delta gives \(2.40\exp(-0.288)\approx2.40(0.750)=1.80\) events per person-year. The delta is dimensionless because it is on the log-rate scale; the shifted prediction has units of events per person-year. Treating the raw difference, \(-0.600\) events per person-year, as a log-rate delta would be incorrect.

What Makes External Information Relevant?

A numerical translation is only as useful as its assumptions. Before using an external estimate to anchor a delta, assess the following points:

Target population and setting. The external participants should represent, or be adjustable to, the people whose outcomes are missing. Differences in care, calendar time, eligibility, or follow-up can alter the contrast.
Covariate comparability. A contrast conditional on \(X\) is generally more relevant than a crude group difference. If the groups differ in predictors of the outcome, an unadjusted marginal contrast may not describe the conditional shift required by the imputation model.
Measurement alignment. Confirm that outcome definitions, instruments, units, timing, and exposure denominators agree. Any conversion between scales needs an explicit and defensible relationship.
Selection and follow-up. A validation sample can be biased if participation or successful measurement depends on the outcome in ways not accounted for. Check how people entered the sample and whether their outcomes were observed under conditions relevant to the target.
Statistical uncertainty. Small samples can yield imprecise contrasts. Report the estimate and its uncertainty on the delta scale where possible, and distinguish that sampling uncertainty from additional uncertainty about transport and measurement assumptions.

A short justification can often be written as a chain: the external source estimates a specified contrast; the contrast is relevant to the target after stated adjustments; the scale conversion is justified; and the resulting delta is used as a central calibration value or as an anchor for a plausible range. If any link is weak, the value should be treated as a less reliable anchor rather than as a precise estimate.

Deriving the Logit-Scale Translation

The binary example used the link-scale definition. The probability implied by a logit delta follows directly from that definition. Let \(p_1\) be the event probability predicted for an observed outcome at a fixed covariate profile, and let \(p_0\) be the assumed probability for a missing outcome. By definition,

\operatorname{logit}(p_0)=\operatorname{logit}(p_1)+\delta.

Exponentiating both sides converts log odds to odds:

\frac{p_0}{1-p_0} =\frac{p_1}{1-p_1}\exp(\delta).

Write the right-hand side as \(A\). Since \(p_0/(1-p_0)=A\), multiplication by \(1-p_0\) gives \(p_0=A(1-p_0)\), so \(p_0(1+A)=A\). Therefore,

p_0=\frac{A}{1+A} =\frac{p_1\exp(\delta)} {1-p_1+p_1\exp(\delta)}.

This shows why a logit delta is not a constant probability difference: the probability shift depends on \(p_1\). It also provides a direct check on a proposed binary delta. A positive delta raises the odds and probability; a negative delta lowers them, for any \(0<p_1<1\).

Calibration Is an Anchor, Not Identification

An external estimate does not, by itself, establish the distribution of outcomes that are missing in the target data. Even a well-conducted validation sample may differ from the target population, and an observational external contrast can remain confounded or selected. Calibration makes an assumption more evidence-informed; it does not remove the need to examine sensitivity to that assumption.

There are several useful comparisons. A zero delta corresponds to the no-shift reference on the chosen model scale. Subject-matter elicitation can supply plausible values when direct external evidence is unavailable, while validation data can give quantitative anchors when relevant outcomes are observed. A range of delta values can reflect uncertainty or disagreement among sources rather than pretending one estimate is exact. These approaches are complementary: external evidence can inform an elicited range, and expert judgment can help assess whether an external estimate transports.

Diagnostics should focus on the calibration chain. Check whether the external data cover the target covariate profiles, whether adjusted and unadjusted contrasts differ substantially, whether the outcome scale conversion is supported, and whether multiple credible sources agree. Investigate incompatible outcome definitions, sparse validation data, or a source whose sampling mechanism selects unusually healthy or severe cases. If there is little covariate overlap or no defensible measurement link, report that limitation and avoid presenting a precise calibrated delta.

A frequent mistake is to treat an external crude difference as the delta merely because it has the same outcome label. For example, if a validation group has a lower mean because it contains older participants, while the imputation model conditions on age, the crude difference mixes a compositional difference with a conditional missing-versus-observed shift. The correction is to align the estimand: compare groups within relevant covariate profiles or use an appropriate adjustment, then express that contrast on the model scale. If this cannot be done, the crude estimate may still inform a broad sensitivity discussion, but it is not a direct calibration of the conditional delta.

Key takeaway: Calibrate a delta by matching the external contrast to the target population and conditional estimand, translating it to the imputation model’s scale, and carrying forward the limitations and uncertainty of that evidence. A calibrated value is a transparent sensitivity assumption, not proof that the missing outcomes follow that shifted distribution.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

An external validation study estimates that the missing group’s conditional mean is 3.5 units below the observed group’s mean on the same continuous outcome scale. What is the additive delta, and what does its sign mean?
For a binary outcome, the observed-group probability is 0.20 and an external source suggests a missing-group probability of 0.10. Explain why the delta is not \(-0.10\), and give the logit-scale delta.
A count study reports 50 events in 20 person-years for one group and 60 events in 20 person-years for a comparable reference group. Calculate the log-rate delta for the first group relative to the reference.
Why may an external crude mean difference fail to calibrate a delta when the imputation model adjusts for covariates?

Solutions. (1) \(\delta=-3.5\) outcome units, indicating an assumed conditional mean 3.5 units lower for missing outcomes. (2) The probability difference is \(-0.10\), but the logit delta is \(\log(0.10/0.90)-\log(0.20/0.80)=\log(4/9)\approx-0.811\). It is on the log-odds scale, not the probability scale. (3) The rates are \(50/20=2.50\) and \(60/20=3.00\) events per person-year, so \(\delta=\log(2.50/3.00)=\log(0.8333)\approx-0.182\). (4) The crude difference can reflect differences in covariate composition as well as the conditional missing-versus-observed contrast. The external comparison must be aligned or adjusted for the predictors used by the imputation model.