Two Sources of Uncertainty in a Population
A dose may produce different outcomes in different people even if the dose-response model parameters were known exactly. This between-subject variability is distinct from uncertainty about the model parameters themselves. A population target-attainment probability must account for the first; uncertainty about its estimated value must account for the second.
For example, a decision maker might ask what fraction of a defined target population, at a given dose, is predicted both to meet an efficacy threshold and to remain below a safety threshold. That fraction is a population quantity conditional on the model parameters and the population distribution. Since the parameters are estimated, the fraction itself has a posterior distribution. These are related but different quantities: variability describes differences among individuals, while parameter uncertainty describes uncertainty in the population prediction.
The definition requires a clearly specified target population, a model for its between-subject variation, and a joint model for efficacy and safety outcomes. If the population also has covariates, their distribution or standardization weights must be specified. In this tutorial, that population definition and its weights are treated as fixed; uncertainty about the covariate mix is a separate issue.
A posterior mean population attainment probability is \(\bar q(d)=\int q(d;\theta)\pi(\theta\mid\text{data})\,d\theta\), where \(\pi(\theta\mid\text{data})\) is the posterior distribution of the parameters. This average is useful, but it does not replace the posterior distribution of \(q(d;\theta)\). A posterior interval for \(q\) describes uncertainty in the population fraction, not the range containing a specified fraction of individual outcomes.
Calculating the Fraction at Fixed Parameters
At a fixed parameter value, the joint probability is most directly computed by integrating the joint outcome distribution over the region that meets both targets. Equivalently, if the outcomes depend on an individual's random effects \(B\), it can be written as an average of individual conditional success probabilities over \(G_\theta\). This integration is the step that converts an individual-level model into a population fraction.
If efficacy and safety outcomes are conditionally independent for a randomly selected individual at fixed \(d\) and \(\theta\), then the joint probability factors into the two marginal probabilities. This is an assumption, not an automatic property of having separate efficacy and safety models. Shared random effects or other dependence can invalidate the factorization; in that case the joint distribution must be used directly.
Worked Example: Integrating Individual Variability at One Dose
Suppose, hypothetically, that at dose \(d=50\) mg and fixed model parameters, an individual's efficacy outcome \(Y_E\) has a normal population distribution with mean 11.5 units and standard deviation 2.0 units. The efficacy target is at least 10 units. The individual's safety burden \(Y_S\) has a normal population distribution with mean 7.0 units and standard deviation 2.4 units; the safety limit is 8.0 units. Assume the two outcomes are independent across individuals' joint population distribution at this dose and parameter value. These summaries describe between-subject distributions, not uncertainty in the means.
For efficacy, standardization gives \(P(Y_E\ge10)=P\left(Z\ge(10-11.5)/2.0\right)=P(Z\ge-0.75)=\Phi(0.75)\approx0.7734\), where \(Z\sim N(0,1)\) and \(\Phi\) is its cumulative distribution function.
For safety, \(P(Y_S\le8)=P\left(Z\le(8-7.0)/2.4\right)=\Phi(0.4167)\approx0.6615\). Under the stated independence assumption,
Thus, at these fixed parameters, the model predicts that about 51.16% of the target population meets both thresholds. It does not predict that every individual has a 51.16% chance of meeting the targets conditional on their own characteristics; rather, it is the population fraction under the specified distribution. Nor does this calculation yet express uncertainty about the model parameters.
The factorization above is justified only by the stated independence assumption. If efficacy and safety outcomes are dependent, multiplying their marginal probabilities can overstate or understate the joint probability. For instance, a shared random effect could make people with higher efficacy response also more likely to have a high safety burden. With dependence, use the joint distribution or simulate both outcomes together for each individual.
Propagating Parameter Uncertainty Separately
Parameter uncertainty is propagated by evaluating the population probability at posterior parameter draws. For each draw, integrate over the subject-level distribution, then summarize the resulting values across draws. The order matters conceptually: each parameter draw defines a population-level probability; the posterior distribution of those probabilities represents uncertainty about that population quantity.
A useful justification follows from iterated expectation. Let \(I\) equal 1 if a randomly selected individual meets both targets and 0 otherwise. At fixed \(\theta\), \(E(I\mid d,\theta)=q(d;\theta)\). Averaging this conditional expectation over the posterior gives
Therefore, the posterior mean of the conditional population fraction equals the posterior predictive probability that a future randomly selected individual from the specified population meets both targets. This identity does not make the two interpretations interchangeable in every report: the posterior predictive probability is one summary, while the distribution of \(q(d;\theta)\) additionally describes parameter uncertainty in the population fraction.
Worked Example: A Posterior Distribution for the Population Fraction
For a transparent hypothetical calculation, suppose the posterior is represented by two equally weighted parameter draws at dose 50 mg. At the first draw, the population efficacy and safety probabilities are 0.7734 and 0.6615, so the conditional joint attainment probability is \(q^{(1)}=0.5116\) (using the unrounded product, 0.5116041). At the second draw, suppose the corresponding probabilities are both 0.8413, with conditional independence.
The second draw gives \(q^{(2)}=(0.8413)(0.8413)=0.70778569\approx0.7078\). The posterior mean, using the unrounded products, is
In this deliberately small, discrete illustration, the posterior distribution assigns probability 0.5 to each of 0.5116 and 0.7078. It therefore shows substantial parameter uncertainty in the population fraction, even though the population variability has already been integrated within each draw. With realistic posterior simulation, many draws would be used and posterior quantiles would usually be more informative than this two-point illustration.
The mean 0.6097 can be interpreted as the posterior predictive probability of joint attainment for a future individual drawn from the defined population. It is not a claim that the true fraction has probability 0.6097 of being attained; that mixes up the fraction with an individual's binary outcome.
Nested Simulation Across Doses
When the integral over random effects has no convenient closed form, nested simulation gives a direct estimate. For each posterior draw, simulate individuals from the model's target-population distribution and simulate their efficacy and safety outcomes jointly. Count the fraction meeting both thresholds. Repeat across posterior draws and across candidate doses. If outcomes are generated conditionally on random effects, retain the same simulated individual's random effects for both outcomes so that any modeled dependence is preserved.
For posterior draw \(m\), sample \(\theta^{(m)}\) from the fitted posterior distribution.
At dose \(d\), simulate \(K\) individuals using the random-effect and outcome distributions conditional on \(\theta^{(m)}\).
Compute \(q^{(m)}(d)=K^{-1}\sum_{k=1}^{K} I_{mk}(d)\), where \(I_{mk}(d)=1\) only when individual \(k\) meets both targets.
Use the \(q^{(m)}(d)\) values to estimate the posterior mean and uncertainty interval for the population fraction at that dose.
Worked Example: Comparing Two Candidate Doses
Consider hypothetical fixed-parameter population calculations with independent efficacy and safety outcomes. At dose 25 mg, suppose the standardized efficacy probability is \(\Phi(0.50)=0.6915\), and the probability of remaining below the safety limit is \(\Phi(1.50)=0.9332\). The joint population fraction is
At 50 mg, use the distributions in the first worked example: \(q(50;\theta)=0.5116\). If the decision rule requires at least 60% of the population to meet both targets, the 25 mg dose meets that criterion because \(0.6453\ge0.60\), while the 50 mg dose does not because \(0.5116<0.60\). The higher dose has a larger marginal efficacy probability in this illustration, but the lower probability of meeting the safety threshold more than offsets that gain in the joint fraction.
This comparison is conditional on fixed parameters. With posterior uncertainty, repeat the calculation at both doses for every parameter draw. A dose might have a posterior mean above 0.60 while a substantial part of its posterior distribution lies below 0.60; reporting only the mean would conceal that uncertainty. The probability \(P(q(d;\theta)\ge0.60\mid\text{data})\) is a posterior probability about the population fraction exceeding the criterion, not the same quantity as \(\bar q(d)\).
Diagnostics, Alternatives, and Limitations
The main diagnostic question is whether the simulated or integrated quantity really corresponds to the intended population fraction. Check that the random effects reproduce the assumed target-population variability, that outcomes for a person are generated jointly under the intended dependence structure, and that the threshold directions are correct. For example, efficacy might require exceeding a lower bound while safety requires staying below an upper bound; reversing either inequality changes the target event.
Monte Carlo error should be assessed separately at the two levels. For a fixed parameter draw and \(K\) independent simulated individuals, the conditional fraction estimate has approximate simulation standard error \(\sqrt{q^{(m)}(d)(1-q^{(m)}(d))/K}\). Increasing \(K\) reduces error in estimating each \(q^{(m)}(d)\). Increasing the number of posterior draws improves summaries of parameter uncertainty. Increasing one without the other does not fix inadequate precision at the other level.
A direct numerical integration over random effects can be more efficient than inner simulation when the distributions are simple and low-dimensional. Nested Monte Carlo is flexible for nonlinear outcomes and complicated joint distributions, but it can be computationally expensive. A shortcut based on marginal efficacy and safety probabilities is valid only under conditional independence; it is not a general alternative to the joint calculation.
- Pooling all simulated indicators can hide uncertainty. It may estimate a posterior predictive probability, but it does not by itself show how the population fraction varies across parameter draws. Preserve and summarize the \(q^{(m)}(d)\) values.
- Ignoring between-subject variability changes the estimand. Plugging in an average individual can estimate attainment for that representative profile, not the fraction of a heterogeneous population attaining the targets.
- Assuming independence without justification can misstate joint attainment. Use a joint model or joint simulation if efficacy and safety outcomes share random effects or otherwise depend on one another.
- Calling posterior uncertainty “population variability” is misleading. A wide posterior interval for \(q(d;\theta)\) reflects uncertainty about the population fraction; a broad distribution of outcomes among individuals contributes to each fixed-parameter \(q(d;\theta)\).
These estimates depend on the correctness of the response and random-effect distributions, the definition of the target population, and the clinical thresholds. They also describe a modeled fraction, not a guarantee about the outcomes of a finite future cohort. If a future cohort has \(N\) independently sampled individuals and the fixed-parameter attainment probability is \(q\), its observed attaining fraction is random; under independent identical sampling, the number attaining both targets follows a binomial distribution with parameters \(N\) and \(q\). That sampling variation is distinct from both individual outcome variability and parameter uncertainty.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- At fixed parameters, the efficacy target probability is 0.80 and the safety probability is 0.75. Under independence, what is the joint attainment probability? What assumption is needed?
- In a posterior simulation, each parameter draw has a corresponding fraction estimated after simulating 10,000 individuals. Which source of uncertainty is reduced by increasing the number of individuals per draw, and which by increasing the number of posterior draws?
- Why is the mean response of a representative individual generally insufficient to calculate a population target-attainment fraction?
- Suppose the posterior mean of \(q(d;\theta)\) is 0.70. Does this mean the posterior probability that the true population fraction exceeds 0.70 is 1? Explain.
Solutions. (1) The joint probability is \((0.80)(0.75)=0.60\), provided efficacy and safety attainment are independent at the fixed dose and parameters. (2) More simulated individuals reduce Monte Carlo error in each conditional population fraction; more posterior draws improve estimation of the posterior distribution across parameter uncertainty. (3) A representative mean profile does not average the threshold indicator over the distribution of individual outcomes and random effects; nonlinear thresholding means the response at the mean need not equal the population fraction. (4) No. A posterior mean of 0.70 does not determine the probability that \(q(d;\theta)>0.70\); that requires the posterior distribution of \(q\), and the probability could be below, equal to, or above 0.5 depending on its shape.