Why the Covariate Mix Matters
A dose can perform differently across covariate groups. For example, a patient characteristic may be associated with lower efficacy, greater toxicity, or both. To predict outcomes for a target population, group-specific predictions must be combined according to the proportions of those groups in that population. If those proportions are uncertain, then the population-level predictions—and potentially the selected dose—are uncertain too.
The earlier material on population heterogeneity in target-attainment probabilities establishes the basic weighted-average idea. Here the focus is the additional uncertainty that arises when the target population’s covariate distribution is not known exactly. This is distinct from parameter uncertainty: even if the model’s group-specific predictions were known without error, uncertain population proportions would still make the aggregate prediction uncertain.
The safety quantity here is defined as the probability of meeting the specified safety criterion; a different safety endpoint can be represented by defining its stratum-specific probability accordingly. These weighted averages describe a randomly selected member of the target population under the stated covariate mix. They are not, by themselves, probabilities that the whole population will experience an event.
The weights must represent the population to which the dose recommendation will apply. Strata should be defined consistently between the source data and the prediction model, and their proportions must sum to one. In practice, uncertainty in \(w_k\) may be represented by a posterior distribution, a sampling distribution, or a set of plausible distributions. The method should match how the weights were obtained.
Propagating Uncertainty in the Weights
Suppose the stratum-specific probabilities are treated as fixed for the moment, while the vector of population proportions \(W=(W_1,\ldots,W_K)\) is uncertain. The aggregate efficacy probability is then itself a random quantity, \(Q_E(d)=\sum_k W_kq_{E,k}(d)\); the same applies to safety. A common Bayesian representation for uncertain proportions is a Dirichlet distribution. For positive parameters \(\alpha_1,\ldots,\alpha_K\), it has mean proportions \(p_k=\alpha_k/A\), where \(A=\sum_k\alpha_k\).
The Dirichlet representation is appropriate when the strata form an exhaustive, mutually exclusive set and the information about their proportions can reasonably be represented by a multinomial sample together with a Dirichlet prior. For example, with prior \(W\sim\operatorname{Dirichlet}(\alpha_1,\ldots,\alpha_K)\), observing stratum counts \(n_1,\ldots,n_K\) gives posterior parameters \(\alpha_k+n_k\). This conjugate update does not correct for a biased sampling frame: a convenience sample can produce precise but irrelevant weights.
The variance formula follows from the Dirichlet moments. In particular, \(E(W_k)=p_k\), \(\operatorname{Var}(W_k)=p_k(1-p_k)/(A+1)\), and \(\operatorname{Cov}(W_j,W_k)=-p_jp_k/(A+1)\) for \(j\ne k\). Expanding the variance of the weighted sum gives
The numerator is the variance of the stratum-specific \(q_k\) values when a stratum is drawn using probabilities \(p_k\). Thus weight uncertainty matters most when the strata differ substantially in their predictions and the effective information about the population proportions is limited. If every stratum has the same \(q_k\), the aggregate is unchanged by the weights and its variance is zero.
Worked Example: Weighting Predictions by a Target Population Mix
Suppose a hypothetical model gives efficacy-target probabilities of 0.86, 0.70, and 0.45 for three strata, and safety-criterion probabilities of 0.96, 0.90, and 0.78. The assumed target population proportions are 0.55, 0.30, and 0.15. Calculate both aggregate probabilities.
For efficacy, the population probability is the weighted sum \(Q_E=0.55(0.86)+0.30(0.70)+0.15(0.45)\). The contributions are \(0.473\), \(0.210\), and \(0.0675\), so \(Q_E=0.7505\), or about 75.1%.
For safety, \(Q_S=0.55(0.96)+0.30(0.90)+0.15(0.78)\). The contributions are \(0.528\), \(0.270\), and \(0.117\), so \(Q_S=0.915\), or 91.5%.
These calculations are conditional on the specified proportions. Equal weighting instead would give efficacy \((0.86+0.70+0.45)/3=0.670\) and safety \((0.96+0.90+0.78)/3=0.880\). Those are different target-population predictions; equal weights are justified only if the target population actually has equal stratum proportions.
Worked Example: Quantifying Uncertainty in a Weighted Prediction
Consider the same three strata, but suppose the covariate proportions have posterior distribution \(\operatorname{Dirichlet}(6,3,1)\). For the efficacy probabilities \(q=(0.85,0.70,0.40)\), calculate the posterior mean and standard deviation of the aggregate efficacy probability, accounting for weight uncertainty only.
Here \(A=6+3+1=10\), so the mean proportions are \(p=(0.6,0.3,0.1)\). The posterior mean aggregate is \(E(Q_E)=0.6(0.85)+0.3(0.70)+0.1(0.40)=0.510+0.210+0.040=0.760\).
For the variance formula, \(\sum_kp_kq_k^2=0.6(0.85^2)+0.3(0.70^2)+0.1(0.40^2)=0.4335+0.1470+0.0160=0.5965\). Therefore, \(\operatorname{Var}(Q_E)=(0.5965-0.760^2)/(10+1)=0.0189/11=0.001718\). The posterior standard deviation is \(\sqrt{0.001718}\approx0.04145\), or about 4.15 percentage points.
This is a measure of uncertainty in the aggregate probability induced by uncertainty in the mix. It is not a confidence interval for an individual patient’s outcome, and it does not include uncertainty in the efficacy model parameters. A posterior interval can be obtained by calculating \(Q_E\) for each posterior draw of \(W\) and taking the desired posterior quantiles.
From Population Uncertainty to Dose Choice
One decision rule is to require the population-level efficacy probability and safety probability each to exceed prespecified targets with sufficiently high probability under the uncertainty in the mix. For example, at dose \(d\), define the separate conditions \(P\{Q_E(d)\geq c_E\mid\text{data}\}\geq\gamma_E\) and \(P\{Q_S(d)\geq c_S\mid\text{data}\}\geq\gamma_S\). Here \(c_E\) and \(c_S\) are the required population probabilities, and \(\gamma_E\) and \(\gamma_S\) are the required levels of confidence in meeting those requirements.
These are separate marginal constraints. Satisfying each at its stated level does not establish that both constraints hold simultaneously at either level; evaluating their joint probability requires additional information about dependence and is a separate issue. The decision rule also needs an objective among doses that pass—for example, selecting the lowest passing dose or minimizing a defined loss.
Worked Example: A Dose Choice That Changes Under Mix Uncertainty
Consider a hypothetical two-stratum population. Let \(w\) denote the high-risk stratum’s population proportion, so the other stratum has proportion \(1-w\). A model produces 1,000 posterior draws of \(w\): 500 draws equal 0.30 and 500 equal 0.70. The two values simplify the arithmetic and represent uncertainty about the covariate mix; a real analysis would use the available posterior draws and assess Monte Carlo precision.
At a low dose, suppose the stratum-weighted population probabilities reduce to \(Q_E=0.82-0.20w\) and \(Q_S=0.98-0.10w\). At \(w=0.30\), these are \(0.82-0.060=0.760\) and \(0.98-0.030=0.950\). At \(w=0.70\), they are \(0.82-0.140=0.680\) and \(0.98-0.070=0.910\).
Set the required population targets to \(c_E=0.72\) and \(c_S=0.93\), with each chance constraint requiring probability at least \(\gamma=0.80\). Among the 1,000 draws, the low dose meets the efficacy target in 500 draws, so the estimated efficacy chance is \(500/1000=0.500\). It meets the safety target in 500 draws as well, giving an estimated safety chance of 0.500. It fails both 0.80 requirements.
At the posterior mean mix \(E(w)=0.50\), however, the low-dose predictions are \(Q_E=0.82-0.20(0.50)=0.720\) and \(Q_S=0.98-0.10(0.50)=0.930\). A rule that checks only the mean mix would call both targets just met. That plug-in conclusion hides the fact that each target is met in only half of the mix draws.
Suppose a higher dose has \(Q_E=0.94-0.10w\) and \(Q_S=0.96-0.04w\). At \(w=0.30\), the probabilities are 0.910 and 0.948; at \(w=0.70\), they are 0.870 and 0.932. Both targets are met in all 1,000 draws, so both estimated chance constraints equal 1.000. Under the stated rule, the higher dose passes and the low dose does not. This conclusion is conditional on the assumed model predictions, target values, posterior draws, and dose options; the example does not establish that a higher dose is generally preferable.
Practical Workflow, Diagnostics, and Alternatives
Specify the strata, the target population, and the source of information about its proportions. Check that the strata are exhaustive, mutually exclusive, and consistently defined.
Represent uncertainty using an appropriate posterior or sampling distribution. Investigate whether the data source is representative; uncertainty modelling cannot remove selection bias by itself.
For each dose and each draw of the weights, calculate the weighted efficacy and safety probabilities. If model parameters are also uncertain, draw them according to the specified uncertainty model and use the corresponding stratum-specific predictions.
Estimate each marginal chance constraint from the fraction of draws meeting its threshold, then apply the prespecified rule for choosing among eligible doses.
Repeat calculations with more draws, alternative defensible population-mix assumptions, and plausible stratum definitions. Report whether the chosen dose changes.
A direct diagnostic is to compare the stratum-specific predictions with the uncertainty in their weights. If predictions differ little across strata, even sizeable weight uncertainty may have little effect on the aggregate. Conversely, if predictions vary sharply and some strata are rare or poorly represented, the aggregate and dose choice may be sensitive to the mix. Inspect the posterior distribution of \(Q_E(d)\) and \(Q_S(d)\), not only their means, and verify that the resulting values remain between zero and one.
Monte Carlo error is another failure mode. If a chance constraint is estimated by the fraction of \(M\) draws that pass, that fraction is itself noisy; near a decision threshold, increase \(M\) and check whether the decision remains stable. Also check that the posterior draws reflect the intended uncertainty: treating estimated proportions as fixed, or drawing from an inappropriate sampling model, can make the apparent precision misleading.
The main alternative to propagating mix uncertainty is a plug-in calculation using estimated proportions, such as the posterior mean. It is simple and can summarize the central prediction, but it does not show how often a population target is met across plausible mixes. Scenario analysis instead evaluates selected plausible mixes, which is transparent and useful when there is no defensible probability distribution, but its conclusions depend on which scenarios are chosen. Robust rules can require acceptable performance across a specified range of mixes; they avoid assigning probabilities to that range but may be conservative.
When model parameters are uncertain as well as the weights, the prediction should propagate both sources rather than silently holding one fixed. If they are assumed independent, draws may be combined independently; if that assumption is not justified, their dependence must be represented. The resulting uncertainty should still be described by source. This tutorial’s separate efficacy and safety chance constraints do not determine the probability that both hold together.
A final limitation is that uncertainty in the population mix is not the same as uncertainty about which covariates matter or whether the model transports to the target population. A model can have well-estimated weights and still be inappropriate if important effect modifiers are absent or the stratum-specific predictions do not apply. Sensitivity analyses can reveal dependence on these assumptions, but cannot substitute for relevant data and model validation.
Common Pitfall
Check Your Understanding
Try each problem, then compare your reasoning with the solutions below.
- A target population has two strata in proportions 0.70 and 0.30. Their efficacy-target probabilities at a dose are 0.80 and 0.50. What is the aggregate efficacy probability?
- If every stratum has the same safety probability, what is the effect of uncertainty in the stratum proportions on the aggregate safety probability? Explain.
- A dose meets an efficacy target in 840 of 1,000 posterior weight draws. What is the estimated efficacy chance constraint probability, and does it meet a required level of 0.90?
- Give one reason why posterior uncertainty in covariate proportions might not adequately represent uncertainty about the target population.
Solutions. (1) \(Q_E=0.70(0.80)+0.30(0.50)=0.56+0.15=0.71\), so the aggregate efficacy probability is 0.710, or 71.0%. (2) If every stratum has safety probability \(q\), then \(Q_S=\sum_kW_kq=q\sum_kW_k=q\), because the weights sum to one. The aggregate is unchanged by mix uncertainty. (3) The estimated probability is \(840/1000=0.840\), which is below 0.90, so the dose fails that chance constraint. (4) For example, if the source sample is not representative of the target population, its posterior proportions may be precise but biased for the target. Uncertainty calculations based on that sample do not automatically correct the mismatch.