Missing Outcomes in Historical Control Data

Why Missing Historical Outcomes Change the Borrowing Model

A power prior borrows information by raising a historical likelihood to a discount power. That operation does not recover outcomes that were never recorded. The likelihood must first represent what the historical data actually reveal, with missing outcomes handled under stated assumptions. Otherwise, missing records can be mistaken for observed failures, or the observed cases can be treated as representative when the missingness pattern makes that implausible.

Let \(Y_i\) denote the outcome for historical control participant \(i\), \(X_i\) any observed covariates, and \(R_i=1\) if the outcome is observed and \(R_i=0\) if it is missing. Write \(D_H\) for the historical data, including observed outcomes, covariates, and missingness indicators. In a borrowing analysis, the historical likelihood contributes information about the outcome-model parameter \(\theta\); a discount \(a\), usually between zero and one, controls how much that contribution is used.

Definition: The observed-outcome likelihood is the probability model for the outcomes that were observed, conditional on the information used in the outcome model. When outcomes are missing, it is obtained by summing or integrating the full outcome model over the unobserved outcomes. A power prior based on this likelihood has the form \(\pi(\theta\mid D_H,a)\propto L_{\mathrm{obs}}(\theta;D_H)^a\pi_0(\theta)\), with a normalizing constant when the powered prior is proper.

The key issue is not whether the missing outcomes are inconvenient; it is whether the observed-data likelihood is adequate for inference about \(\theta\). That depends on how outcomes became missing and on the assumptions connecting the missingness and outcome processes.

Observed Outcomes and the Power Prior

For independent Bernoulli outcomes with a common historical control event probability \(p\), let \(n_{\mathrm{obs}}\) outcomes be observed, with \(s\) events and \(f=n_{\mathrm{obs}}-s\) non-events. The historical outcome likelihood, apart from a factor not depending on \(p\), is

L_{\mathrm{obs}}(p)\propto p^s(1-p)^f.

If the fixed power is \(a\), the historical contribution is \(p^{as}(1-p)^{af}\). For a Beta prior \(\mathrm{Beta}(\alpha_0,\beta_0)\), this gives a historical-only powered posterior \(\mathrm{Beta}(\alpha_0+as,\beta_0+af)\). These fractional exponents are the consequence of discounting; they do not mean that fractional participants were observed.

Let \(L_C(\theta;D_C)\) be the likelihood from the current trial. With a fixed \(a\), the posterior is proportional to \(L_C(\theta;D_C)L_{\mathrm{obs}}(\theta;D_H)^a\pi_0(\theta)\). The current data are not discounted by this historical power. If missingness is nonignorable and modeled jointly, the historical contribution may instead involve a joint observed-data likelihood for outcomes and missingness; the model and its parameters must be specified before deciding what to power.

Worked Example: Binary Outcomes With Missing Records

Suppose a hypothetical historical control group has 100 participants. Outcomes are recorded for 80: 48 had the event and 32 did not. The remaining 20 outcomes are missing. Use a historical discount \(a=0.5\) and a baseline prior \(p\sim\mathrm{Beta}(2,2)\). For this example, assume a common event probability and an ignorable missingness process, so that the observed outcomes supply the outcome likelihood.

The observed likelihood is \(L_{\mathrm{obs}}(p)\propto p^{48}(1-p)^{32}\). Raising it to \(0.5\) gives \(p^{24}(1-p)^{16}\). Thus, the historical-only powered posterior is

p\mid D_H,a \sim \mathrm{Beta}(2+24,\,2+16)=\mathrm{Beta}(26,18).

Its mean is \(26/(26+18)=26/44=0.5909\). This is a posterior mean event probability of about 59.1% under the stated model and prior. It does not treat the 20 missing outcomes as either events or non-events.

For contrast, coding all 20 missing outcomes as non-events would produce a powered posterior \(\mathrm{Beta}(26,28)\), with mean \(26/54=0.4815\). That is a different, generally unjustified data assumption—not a correction for missingness. If the outcomes were missing because event participants were more likely to be lost, the ignorable analysis could also be biased; the observed counts alone cannot settle that question.

When Can the Missingness Mechanism Be Ignored?

A common formal classification is based on the relationship between \(R\), the missingness indicators, and \(Y_{\mathrm{mis}}\), the unobserved outcomes. Missing completely at random (MCAR) means that the missingness distribution does not depend on the outcomes or covariates. Missing at random (MAR) means that, conditional on observed outcomes and covariates, the missingness distribution does not depend further on the missing outcomes. Missing not at random (MNAR) means that missingness still depends on unobserved outcomes after conditioning on the observed information.

MAR is a statement about the data-generating mechanism, not a pattern that can generally be confirmed from observed data. In particular, a similar observed event rate among people with and without missing outcomes does not prove MAR: the outcomes of those with missing records are unavailable for that comparison.

For Bayesian inference, ignorability needs more than a MAR label. The outcome and missingness parameters must be distinct, and their prior distribution must factor independently. Under those conditions, and with a correctly specified outcome model, the missingness factor does not alter the posterior for the outcome parameter. If the priors are dependent, information about the missingness parameter can update the outcome parameter through that dependence.

Key condition: A missingness mechanism can be ignored for Bayesian inference about \(\theta\) when the mechanism is MCAR or MAR, the outcome and missingness parameters are distinct, the prior factors independently as \(\pi(\theta,\psi)=\pi(\theta)\pi(\psi)\), and the outcome model is correctly specified. If these conditions do not hold, an explicit model or a justified alternative analysis is needed.

Complete Cases, Covariates, and the Target Population

A complete-case analysis retains records with observed outcomes and any other variables required by the analysis. When only the outcome can be missing, this amounts to analyzing the observed outcomes. For a common Bernoulli probability, that analysis uses the likelihood \(p^s(1-p)^f\); it does not require filling in each missing value. Under MCAR, observed cases can represent the full group with respect to the modeled outcomes, although fewer observations mean less information. Under MAR conditional on covariates, however, the observed cases may have a different covariate distribution from the historical group or the current trial population.

The remedy may be to model outcomes conditional on the observed covariates and then standardize to a stated target covariate distribution. In that analysis, the likelihood is built from the observed outcomes and their covariates. Merely analyzing the observed records and reporting their unadjusted average does not automatically account for covariate-dependent missingness. If covariates themselves are missing, the definition of complete cases becomes stricter, and excluding records can discard additional information.

Worked Example: Covariate-Dependent Observation and Standardization

Consider a hypothetical historical group with two baseline risk strata, low and high. Outcomes are observed for 60 of 80 low-risk participants, including 12 events, and for 20 of 40 high-risk participants, including 12 events. Assume missingness is MCAR within each stratum, and the outcome probability is constant within a stratum. Suppose the target population for the current trial has equal proportions in the two strata. Use independent \(\mathrm{Beta}(2,2)\) priors for the low- and high-risk event probabilities and a fixed power \(a=0.5\).

The observed low-risk likelihood is proportional to \(p_L^{12}(1-p_L)^{48}\), because 60 outcomes are observed and 48 are non-events. Its powered contribution gives

p_L\mid D_H,a\sim\mathrm{Beta}(2+0.5(12),\,2+0.5(48)) =\mathrm{Beta}(8,26),

with posterior mean \(8/34=0.2353\). For the high-risk stratum, 20 outcomes are observed: 12 events and 8 non-events. Therefore,

p_H\mid D_H,a\sim\mathrm{Beta}(2+0.5(12),\,2+0.5(8)) =\mathrm{Beta}(8,6),

with posterior mean \(8/14=0.5714\). The target-population event probability is the equal-stratum standardized probability, \((p_L+p_H)/2\). Its posterior mean is \((0.2353+0.5714)/2=0.4034\).

Pooling the observed cases without stratification gives \(24/80=0.3000\). That calculation weights the low-risk stratum by \(60/80\) and the high-risk stratum by \(20/80\), reflecting their observed-case composition rather than the stated equal-stratum target. The difference illustrates why missingness that varies by observed covariates can make an unadjusted complete-case summary inappropriate even when missingness is ignorable within covariate strata.

When an Explicit Missing-Data Model Is Needed

If missingness may depend on the unobserved outcome, a model for the outcome alone does not generally identify the full-group outcome distribution from observed outcomes. One approach is a selection model: specify both an outcome model \(f(y\mid x,\theta)\) and a missingness model \(P(R\mid y,x,\psi)\). For binary outcomes, the observed-data likelihood includes the observed outcomes for \(R=1\) and sums over both possible outcomes when \(R=0\). It may be necessary to specify or constrain the missingness parameters using external knowledge or sensitivity assumptions.

A pattern-mixture approach instead models outcome distributions separately across missingness patterns and relates them through assumptions or sensitivity parameters. The choice between these approaches is substantive as well as statistical. Neither makes MNAR behavior identifiable from the observed outcomes alone without additional assumptions. When an explicit joint model is used in a power prior, state whether the historical contribution discounts only the outcome component or the joint observed-data contribution; the choice affects what information is borrowed.

Worked Example: Sensitivity to Outcome-Dependent Missingness

Suppose a hypothetical historical dataset has 100 observed outcomes: 60 events and 40 non-events. Let \(p\) be the event probability in the full historical population. Assume, for illustration, that an event is observed with probability \(\rho_1=0.8\), while a non-event is observed with probability \(\rho_0=0.4\). These specified observation probabilities describe an MNAR mechanism because observation depends on the outcome.

Among observed outcomes, the event probability is

q=P(Y=1\mid R=1) =\frac{p\rho_1}{p\rho_1+(1-p)\rho_0}.

The observed event fraction is \(q=60/100=0.6\). Substituting the assumed observation probabilities and solving,

0.6=\frac{0.8p}{0.8p+0.4(1-p)} \quad\Longrightarrow\quad 0.6(0.4+0.4p)=0.8p \quad\Longrightarrow\quad p=\frac{0.24}{0.56}=\frac{3}{7}=0.4286.

Under these particular assumptions, the implied full-population event probability is 42.9%, not the observed-case fraction of 60%. The result is conditional on the chosen observation probabilities; it is not learned from the observed event fraction alone. The corresponding observed-data likelihood, for \(m\) missing outcomes, is proportional to

(0.8p)^{60}\{0.4(1-p)\}^{40} \{p(1-0.8)+(1-p)(1-0.4)\}^{m}.

The final factor accounts for the probability of a missing record after summing over its unobserved outcome. In a real borrowing analysis, conclusions should be checked across plausible values of the observation probabilities or other MNAR sensitivity parameters. A single assumed mechanism should not be presented as if the data had established it.

Derivation: Why Ignorable Missingness Drops Out

The central factorization explains why the conditions for Bayesian ignorability matter. Let \(\theta\) parameterize the outcome model and \(\psi\) the missingness model. Under MAR, with observed covariates conditioned upon, the observed-data likelihood can be written as an outcome factor times a missingness factor:

L(\theta,\psi;D_H) =L_{\mathrm{obs}}(\theta;Y_{\mathrm{obs}},X)\, L_R(\psi;R,Y_{\mathrm{obs}},X).

If the prior also factors independently, then the joint posterior is proportional to

L_{\mathrm{obs}}(\theta)\,L_R(\psi)\,\pi(\theta)\,\pi(\psi).

To obtain the marginal posterior for \(\theta\), integrate over \(\psi\). The result is proportional to

L_{\mathrm{obs}}(\theta)\pi(\theta) \int L_R(\psi)\pi(\psi)\,d\psi.

The integral is a factor that does not depend on \(\theta\), so it cancels in posterior normalization. The marginal posterior for \(\theta\) is therefore based on \(L_{\mathrm{obs}}(\theta)\pi(\theta)\). If \(\pi(\theta,\psi)\) does not factor, the integral can depend on \(\theta\), and the missingness information need not cancel. This is why distinct parameters alone are not a sufficient Bayesian ignorability condition.

With a fixed power prior under the ignorable model, the historical outcome factor becomes \(L_{\mathrm{obs}}(\theta)^a\). The same reasoning does not justify dropping a nonignorable missingness factor; that requires a joint model and explicit assumptions about its parameters.

Diagnostics, Comparisons, and Limitations

Before borrowing, inspect the amount and pattern of missingness by historical study, calendar period, treatment or control group, and important observed covariates. Compare observed-case covariate distributions with the target trial’s distributions. Look for abrupt changes in follow-up or outcome ascertainment that could signal different missingness processes. These checks can reveal incompatibility or poor overlap, but they cannot verify MAR when missing outcomes are unobserved.

Approach	What it uses	Main requirement or limitation
Observed-outcome analysis	Likelihood from observed outcomes, raised to the historical power	Requires a defensible ignorability assumption and an adequate outcome model
Complete-case analysis	Records complete for all variables in the analysis	Can change the covariate mix and target; may discard usable observed information
Explicit missing-data model	Outcome and missingness components, integrating over unobserved outcomes	Requires additional assumptions; MNAR conclusions can be sensitive to unidentifiable parameters

A practical analysis should report the missing-outcome counts and assumptions, not just the posterior based on observed records. Useful checks include repeating the analysis under alternative plausible missingness assumptions, comparing a pooled analysis with covariate-adjusted or stratified analyses, and examining whether the historical borrowing conclusion changes materially. If the missingness model is uncertain, sensitivity analysis is part of the result, not an optional embellishment.

Pitfall: treating missing as a non-event. A missing record is not an observed non-event. Coding it as one changes the likelihood and requires an explicit, usually unjustified imputation assumption.
Pitfall: assuming MAR because observed groups look similar. Similar observed summaries cannot reveal how missing outcomes differ. MAR is an assumption informed by context and measured covariates.
Pitfall: assuming distinct parameters guarantee Bayesian ignorability. If outcome and missingness parameters have dependent priors, missingness data can still update the outcome parameter.
Pitfall: borrowing an unadjusted complete-case rate for a different target mix. If observation depends on covariates and risk varies across them, standardize or model the covariates rather than silently using the observed-case mix.

Key takeaway: Construct the power prior from the likelihood for the observed historical outcomes under a clearly stated model. Ignorability requires MAR or MCAR, distinct outcome and missingness parameters, and independent prior factors for Bayesian inference. When outcome-dependent missingness is plausible, specify a joint model and examine sensitivity to its assumptions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A historical Bernoulli control dataset has 50 observed outcomes, 15 events and 35 non-events, plus 10 missing outcomes. With fixed power \(a=0.4\), what likelihood contribution is used under an ignorable missingness assumption?
Why does MAR alone not guarantee Bayesian ignorability?
In a stratified analysis, why can the pooled event fraction among observed cases differ from the event probability standardized to the target population?
What additional assumption is needed to use an outcome-dependent selection model to estimate the full-group event probability?

Solutions. (1) The observed likelihood is proportional to \(p^{15}(1-p)^{35}\); its powered contribution is \(p^6(1-p)^{14}\). The 10 missing outcomes do not enter as observed events or non-events. (2) The outcome and missingness parameters must also be distinct and have an independently factoring prior; otherwise, missingness data can update the outcome parameter through prior dependence. The outcome model must be correctly specified as well. (3) Missingness can alter the covariate composition of observed cases. If outcome risks differ by stratum, pooling weights strata by their observed counts, whereas standardization uses the target population’s stratum proportions. (4) The selection probabilities or other parameters linking outcome to observation must be specified, constrained by external information, or varied in a sensitivity analysis. The observed outcomes alone do not identify the MNAR mechanism.