When a Bayesian Model Does Not Match the Estimand

Why Model–Estimand Alignment Matters

A fitted Bayesian model produces a posterior distribution for its parameters. A clinical estimand, however, is a precisely defined treatment-effect target: it specifies the population, treatments being compared, outcome variable, handling of intercurrent events, and summary measure. A model can fit the observed data well and still answer a different question from the one stated in the estimand. The key diagnostic is therefore not only whether the model fits, but whether the desired estimand is a parameter or a well-defined function of the model’s parameters under defensible assumptions.

For example, a regression coefficient conditional on baseline covariates is not automatically a marginal treatment effect for the trial population. Likewise, a model that analyzes only outcomes observed before treatment discontinuation does not automatically estimate an effect under a treatment-policy strategy. The mismatch is about the target, not merely about computational technique.

Write \(Y^a\) for the outcome that would be observed under treatment \(a\), with the outcome and intercurrent-event strategy specified by the estimand. For a marginal mean-difference estimand in a target population, the target could be \(\psi=E(Y^1)-E(Y^0)\). A fitted model has parameters, say \(\theta\), and a posterior distribution \(p(\theta\mid D)\) given data \(D\). If the estimand is a function \(\psi=g(\theta)\) under the model, its posterior is obtained by transforming posterior draws: draw \(\theta^{(m)}\) from \(p(\theta\mid D)\), then calculate \(\psi^{(m)}=g(\theta^{(m)})\). This is justified only if the model and its assumptions connect the observed data to the estimand.

A Diagnostic Sequence

Before interpreting a posterior coefficient, compare the model and estimand along four dimensions. The target population and analysis population need not be identical, but any difference must be justified. The outcome scale and time point must agree. The treatment contrast must use the same summary measure. Finally, the model’s handling of intercurrent events must represent the estimand’s strategy rather than silently replacing it.

This sequence distinguishes three situations. First, the model parameter may directly equal the target, such as a normally modeled treatment-group mean difference when the estimand is that same mean difference and the model is correctly specified. Second, the model may support the target after a transformation, such as standardizing conditional predictions over the target population. Third, the data and model may not identify the target without additional assumptions; relabeling a coefficient cannot solve that problem.

Conditional and Marginal Effects

A conditional treatment effect compares treatment groups at fixed covariate values. A marginal effect averages over a specified population’s covariate distribution. These can differ even when the conditional model is correctly specified. In particular, for nonlinear measures such as odds ratios, averaging risks and then forming an odds ratio generally does not produce the same result as averaging conditional odds ratios. This is not necessarily confounding or model failure; it is a difference in the estimand.

The model can still be useful for a marginal target. For each posterior draw, predict the outcome under each treatment for every member of the target population (or an appropriate representation of its covariate distribution), average those predictions separately, and take the requested contrast. This procedure is often called posterior standardization or g-computation. It produces a posterior distribution for the specified marginal estimand, provided the conditional outcome model is adequate and the target covariate distribution is represented.

Here \(X_i\) are baseline covariates for the target population, and \(\theta^{(m)}\) is posterior draw \(m\). For binary outcomes, the conditional expectations are predicted risks, so their difference is a risk difference. If the target population is not the enrolled sample, the \(X_i\) values or weights must represent the intended target population instead. A posterior transformation cannot correct a target-population mismatch by itself.

Intercurrent-Event Strategies Must Also Match

An intercurrent event, such as treatment discontinuation or rescue medication, can change the outcome being targeted. A treatment-policy strategy uses outcomes regardless of whether the event occurred. A hypothetical strategy targets outcomes that would have occurred under a specified scenario in which the event did not occur. Other strategies may define a composite outcome or a while-on-treatment outcome. These distinctions are established in the earlier material on estimands and intercurrent events; here the concern is whether the fitted analysis actually implements the stated strategy.

Conditioning on event status, excluding participants after an event, or censoring at the event does not by itself define a valid strategy. In particular, participants who discontinue or receive rescue medication may differ prognostically from those who do not. A comparison restricted to those who remain on treatment can therefore differ from the randomized-population treatment-policy contrast. Estimating a hypothetical strategy generally requires assumptions and a model for outcomes that would have occurred under the hypothetical scenario; observed outcomes among participants who avoided the event are not automatically those counterfactual outcomes.

The event strategy must be checked in the likelihood and in the data construction. If outcomes after discontinuation are collected and the target is treatment policy, excluding those outcomes creates a mismatch. If the target is hypothetical no-discontinuation, including post-discontinuation outcomes as if they represented continued treatment may create a different mismatch. If those outcomes are missing, the model’s missing-data assumptions matter as well; missing outcome data and an intercurrent event are not interchangeable concepts.

Justifying the Posterior Transformation

Posterior standardization is not a device for changing the estimand after seeing results. It is a way to calculate a target functional from a model that describes conditional outcome distributions. Its justification can be stated directly. Let \(X\) denote baseline covariates in the target population. Under consistency, treatment assignment conditions that identify the treatment-specific outcome distribution, and an adequate outcome model, the marginal mean under treatment \(a\) is the average of the conditional mean over the target distribution of \(X\):

For each posterior draw, replacing the conditional mean by its model prediction and averaging over the target covariate distribution yields a draw for \(E(Y^a)\). Subtracting the two treatment-specific draws gives a draw for the marginal mean difference. The argument depends on the conditional predictions being valid in the covariate regions being averaged over. In a randomized trial, randomization supports identification of treatment comparisons, but it does not make a misspecified outcome model correct or ensure that sparse covariate patterns are adequately supported.

For an intercurrent-event strategy, the conditional outcome in this argument must itself correspond to that strategy. A treatment-policy prediction must include outcomes after the event as defined by the strategy. A hypothetical prediction must represent the counterfactual outcome under the specified no-event scenario and therefore needs assumptions that identify that counterfactual. The averaging formula does not supply those assumptions.

Diagnostics, Alternatives, and Limitations

Several warning signs suggest a mismatch: the reported quantity has a different scale from the estimand; it is conditional when the estimand is marginal; the analysis population excludes participants because of a post-randomization event; or the model’s outcome is defined only before an event although the target uses outcomes after it. A difference between a regression coefficient and a standardized contrast is not alone proof of poor fit. First establish whether they are different targets. Then assess fit, predictive performance, covariate overlap, and sensitivity to plausible model specifications as relevant to the target.

There are two common routes when the target is marginal. One is a model parameterized directly for a marginal treatment contrast. Another is a conditional model followed by posterior standardization. The first can make the target coefficient easier to interpret, while the second can flexibly account for prognostic covariates and nonlinear outcome relationships. Neither route is automatically superior: the important requirement is that the likelihood and reported posterior quantity support the estimand and its assumptions.

When event handling is incompatible, the remedy depends on the target. For a treatment-policy estimand, use outcomes after the event when collected and account for genuinely missing outcomes. For a hypothetical estimand, use a model and assumptions that address the unobserved counterfactual scenario. A completers-only or event-free analysis may be a descriptive alternative, but it should be labeled as such rather than presented as the original estimand.

A Bayesian prior can regularize uncertain parameters and influence posterior uncertainty, but it cannot turn a conditional effect into a marginal one without the needed transformation. It also cannot identify unobserved counterfactual outcomes solely by being Bayesian. Where the data do not identify the target without assumptions, those assumptions must be stated and their consequences examined. The upcoming analysis of sensitivity across plausible estimands addresses a separate question: how conclusions vary when the target itself changes.

Common Mistake

• Wrong reasoning: “The treatment coefficient is estimated precisely, so it is the clinical treatment effect.”
• Correction: Precision describes uncertainty about the model quantity. First verify that the coefficient is the stated estimand or transform posterior draws into the required contrast. A precise posterior for the wrong target remains a precise answer to a different question.
• Wrong reasoning: “Removing participants who used rescue medication estimates what would have happened without rescue.”
• Correction: Removal conditions on a post-randomization event and may select participants with different prognoses. A hypothetical no-rescue estimand requires assumptions about the counterfactual outcomes of those who did use rescue.