Why the Target Population Changes the Estimand
A treatment effect is not fully specified by naming an outcome and a comparison. It also matters for whom the effect is being summarized. A randomized trial may enroll a population with relatively few patients at high baseline risk, while the clinical population to which a decision will apply contains many more such patients. If treatment effects vary with baseline risk, averaging over these two populations can produce different marginal effects, even when the conditional effects within risk groups are the same.
This is a question about the estimand, not merely about the analysis set. The trial-population estimand averages over the covariate distribution among trial participants. A broader-population estimand averages over a separately specified target distribution. The target should therefore be defined substantively—for example, all eligible patients treated in a health system during a specified period—not chosen only because it gives a preferred numerical result.
The outcome and effect scale must be held fixed when comparing these estimands. For a continuous outcome, \(\tau(x)\) may be a difference in mean outcome, measured in the outcome’s units. For a binary outcome, it could be a conditional risk difference; standardizing the two risks separately also permits a target-population risk ratio. Earlier material on marginal and conditional treatment-effect estimands and on binary-outcome scales supplies the corresponding scale-specific interpretation.
Bayesian Standardization
In a Bayesian analysis, the conditional effects are uncertain. Let \(\theta\) denote the parameters of a model for the potential-outcome means or another suitable outcome summary. For each posterior draw \(\theta^{(m)}\), calculate the conditional effect \(\tau^{(m)}(x)\) and average it over the target covariate distribution:
The resulting draws form the posterior distribution of the target estimand, conditional on the observed data, model, and target-population information. Posterior means, medians, credible intervals, or probabilities about \(\psi_T\) should be computed from these standardized draws. If the target distribution is represented by a finite set of covariate profiles \(x_1,\ldots,x_K\) with weights \(w_1,\ldots,w_K\), where \(w_k\geq0\) and \(\sum_k w_k=1\), the calculation is
The weights describe the target population, not treatment allocation within the trial. They may come from a registry, a health-system database, a population survey, or a prespecified distribution of clinically relevant profiles. If they are estimated rather than treated as known, uncertainty about them can be propagated—for example, by drawing the weights from a model for the target covariate distribution and standardizing each effect draw using a corresponding weight draw.
For the mean statement, write \(\psi_T=\sum_k w_k\tau_k\), with fixed \(w_k\). Linearity of expectation gives \(E(\psi_T\mid D)=\sum_k w_k E(\tau_k\mid D)\). For uncertainty, however,
The covariance terms can be positive or negative. Retaining joint posterior draws automatically incorporates them; constructing an interval from marginal summaries generally does not.
Worked Example: Changing the Covariate Mix Changes the Average
Hypothetical data and model. Consider a continuous clinical outcome measured in points, with lower values better. Patients are classified at baseline as lower risk or higher risk. A Bayesian outcome model provides the following illustrative posterior draws of the treatment-minus-control mean difference in each stratum. For this example, trial participants are 80% lower risk and 20% higher risk; the broader target population is 40% lower risk and 60% higher risk.
| Posterior draw | Lower-risk effect (points) | Higher-risk effect (points) |
|---|---|---|
| 1 | -1 | -7 |
| 2 | -2 | -8 |
| 3 | -3 | -9 |
| 4 | 0 | -6 |
For draw 2, the trial-population average is \(0.80(-2)+0.20(-8)=-1.6-1.6=-3.2\) points. The target-population average for the same draw is \(0.40(-2)+0.60(-8)=-0.8-4.8=-5.6\) points. The arithmetic uses identical stratum-specific effects; only the weights change.
Repeating that calculation for all four draws gives trial-population values \(-2.2,-3.2,-4.2,-1.2\) points, and target-population values \(-4.6,-5.6,-6.6,-3.6\) points. Their posterior means across these four illustrative draws are \(-2.7\) and \(-5.1\) points, respectively. All four draws favor treatment on this scale, but these four values are shown to demonstrate standardization, not to provide a reliable credible interval. A real analysis would use many posterior draws and report uncertainty appropriate to the full posterior.
The more favorable target average arises because the target assigns greater weight to the higher-risk stratum, where the modeled treatment effect is larger in magnitude. It is not evidence that treatment has become more effective within either stratum.
Assumptions for Generalization
Randomization supports comparisons between treatment groups within the trial, subject to the trial’s design and analysis assumptions. It does not, by itself, establish that the conditional effects in trial participants apply to people outside the trial. Generalizing a trial estimate to a target population requires additional assumptions and information.
- Well-defined treatment and outcome. The treatment strategies, outcome, follow-up period, and handling of intercurrent events must have the same intended meaning for trial participants and the target population. The potential outcomes \(Y(1)\) and \(Y(0)\) must correspond to these strategies.
- Conditional transportability. Given the included baseline covariates \(X\), the relevant conditional potential-outcome means—or at least the conditional treatment effects, when only an effect is being transported—must be applicable from trial participants to the target. This assumption fails if an omitted variable both differs between trial and target and modifies the treatment effect.
- Covariate support. Target profiles receiving positive weight must be represented, or adequately supported, in the trial data. For a target profile absent from the trial, its conditional effect cannot be learned from trial outcomes without extrapolating the model.
- Adequate treatment information within the trial. For the conditional effects to be estimated, both treatment strategies must have sufficient randomized representation across relevant covariate profiles. Sparse strata can leave conditional effects weakly informed even when the overall trial is large.
- Suitable outcome and covariate measurement. Important effect modifiers must be measured comparably, and the model must represent the relationships needed for standardization. Measurement differences or severe model misspecification can undermine transport.
These assumptions are partly substantive: observed data cannot prove that all effect modifiers have been measured. They can, however, be made explicit and assessed for plausibility. Standardization changes the population over which effects are averaged; it does not remove bias from an inappropriate conditional model or from a nontransportable effect.
Worked Example: Standardizing Binary Risks
Hypothetical model output. Suppose a Bayesian model yields the following illustrative posterior expected event risks in two baseline strata. These values are used here as one posterior draw’s conditional risks, not as fixed truths. The trial consists of 70% lower-risk and 30% higher-risk patients; the target consists of 30% lower-risk and 70% higher-risk patients.
| Baseline stratum | Risk with treatment | Risk with control |
|---|---|---|
| Lower risk | 0.10 | 0.15 |
| Higher risk | 0.35 | 0.50 |
For the trial covariate mix, the standardized treatment risk is \(0.70(0.10)+0.30(0.35)=0.070+0.105=0.175\), or 17.5%. The standardized control risk is \(0.70(0.15)+0.30(0.50)=0.105+0.150=0.255\), or 25.5%. The trial-mix risk difference is therefore \(0.175-0.255=-0.080\), an 8.0 percentage-point lower event risk with treatment.
For the target mix, the treatment risk is \(0.30(0.10)+0.70(0.35)=0.030+0.245=0.275\), or 27.5%. The control risk is \(0.30(0.15)+0.70(0.50)=0.045+0.350=0.395\), or 39.5%. Thus the target-mix risk difference is \(0.275-0.395=-0.120\), a 12.0 percentage-point lower event risk with treatment. The target-mix risk ratio for this draw is \(0.275/0.395=0.6962\).
In an actual Bayesian analysis, repeat both risk-standardization calculations for every joint posterior draw, then calculate the risk difference or ratio within each draw. In particular, the posterior distribution of a risk ratio is obtained from draw-wise ratios; the ratio of two posterior mean risks need not equal the posterior mean risk ratio.
Support, Diagnostics, and Alternatives
A basic diagnostic is to compare the target and trial distributions of important baseline covariates and effect modifiers. Tabulations by clinically meaningful strata can reveal target profiles missing from the trial or represented by very few participants. In continuous covariates, plots and summaries of the distributions can show where extrapolation would be required. If selection or sampling weights are used, unusually large weights are a warning that a small part of the trial is being asked to represent a large part of the target. Such weights can make estimates highly sensitive to a few observations.
Consider a target population with 48% lower-risk, 32% higher-risk, and 20% very-high-risk patients, while the trial contains no very-high-risk patients. The first two target groups together make up 80% of the target, so a supported-target average can be computed by renormalizing their weights to 60% and 40%. But the average effect for the full target requires the very-high-risk conditional effect as well. A model may generate a value there, but trial data provide no direct information about that stratum; the result depends on extrapolation and modeling assumptions. A defensible analysis should distinguish these two quantities rather than present the extrapolated full-target result as if all target groups were observed.
Worked Example: What Can Be Learned Without Target Support?
Hypothetical setup. Suppose conditional mean treatment effects, in outcome units, are \(-2\) for lower-risk patients and \(-5\) for higher-risk patients. The target population is 48% lower risk, 32% higher risk, and 20% very high risk, but the trial has no very-high-risk participants.
Among the supported portion of the target, the original weights sum to \(0.48+0.32=0.80\). Renormalizing gives lower-risk weight \(0.48/0.80=0.60\) and higher-risk weight \(0.32/0.80=0.40\). The supported-target average is \(0.60(-2)+0.40(-5)=-1.2-2.0=-3.2\) outcome units.
The full-target average would be \(0.48(-2)+0.32(-5)+0.20\tau_{\mathrm{VH}}\), where \(\tau_{\mathrm{VH}}\) is the very-high-risk effect. Since this stratum is absent from the trial, the supplied data do not identify \(\tau_{\mathrm{VH}}\). For example, inserting a model-based value of \(-8\) would give \(-0.96-1.60-1.60=-4.16\) units, but that number relies on the extrapolated value \(-8\), not on observed trial support. The supported average and the full-target, model-extrapolated average answer different questions.
Outcome-model standardization directly predicts conditional outcomes or effects at target covariate profiles and averages them using target weights. It is convenient when the outcome model is credible and the target distribution is available. Weighting methods instead reweight trial participants to resemble the target on measured covariates; they can be useful as a complementary analysis but may become unstable with poor overlap. A conditional-effect summary describes effects by covariate profile rather than answering the population-average question. Restricting the target to the region with trial support can avoid unsupported extrapolation, but it changes the population to which the conclusion applies.
No method makes these approaches interchangeable. Model standardization can be sensitive to model form; weighting can be sensitive to extreme weights and sparse overlap; and a restricted target may not match the clinical decision population. Sensitivity analyses can examine alternative plausible covariate distributions, effect-modifier specifications, or supported target definitions. Their purpose is to show how conclusions depend on choices and assumptions, not to select whichever target gives the most favorable result.
Common Pitfall and Interpretation
A common mistake is to report a target-standardized effect as though it were the effect directly observed among the enrolled participants. For example, the \(-5.6\)-point target average in the first example is not a revised trial estimate: it is the posterior treatment effect averaged over a different covariate distribution, under assumptions that allow the conditional effects to apply there. The trial-population average remains tied to the trial’s covariate mix.
A second mistake is to treat similarity in overall outcome risk as proof that the treatment effect transports. Similar overall risk does not guarantee similar effect-modifier distributions or conditional treatment effects. Conversely, different baseline risk does not automatically preclude generalization if the relevant conditional effects are stable and the target covariates are supported. State the target, scale, covariates used for standardization, support limitations, and assumptions alongside the posterior summary.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A trial is 90% lower-risk and 10% higher-risk. The target is 50% in each stratum. The treatment effects are \(-1\) and \(-5\) points, respectively. Calculate the two population averages.
- Why must a Bayesian analysis standardize each posterior draw rather than average conditional credible interval endpoints?
- A target group has no members represented in the trial. What additional claim is needed to report a full-target effect, and what should be disclosed?
- For binary outcomes, why should a posterior risk ratio generally be calculated within each draw?
Solutions. (1) The trial average is \(0.90(-1)+0.10(-5)=-0.9-0.5=-1.4\) points. The target average is \(0.50(-1)+0.50(-5)=-0.5-2.5=-3.0\) points. (2) The marginal posterior depends on the joint uncertainty and covariance among conditional effects; draw-wise standardization preserves that dependence, whereas separately averaging interval endpoints does not. (3) One must assume or model that the conditional effect extends to the unsupported group. Disclose that the estimate relies on extrapolation, identify the assumptions, and distinguish it from an average limited to supported profiles. (4) A ratio is nonlinear, so the posterior mean of draw-wise risk ratios generally differs from the ratio of posterior mean risks; draw-wise calculation gives the posterior distribution for the ratio estimand.