Two Different Sources of Uncertainty
A prediction for a future person can be uncertain for more than one reason. The human population may vary: two people with the same dose can have different clearances or responses. At the same time, the population distribution itself is not known exactly because it depends on uncertain model parameters. These sources answer different questions. Population variability describes differences among individuals, conditional on a specified population model. Parameter uncertainty describes what is not known about the parameters defining that model.
Nested simulation keeps these layers distinct by drawing parameters first and individuals second. An outer draw represents one plausible set of population parameters; inner draws represent future individuals under that set. The resulting predictions can describe either a randomly selected future individual or a future cohort. The distinction matters: a cohort average can be less variable than an individual outcome, but uncertainty about the population parameters affects the cohort prediction too.
The uncertainty distribution for \(\theta\) may, for example, be a posterior distribution or a bootstrap approximation. The earlier material on Monte Carlo propagation establishes how uncertain inputs can be carried through a prediction; the additional step here is to retain the distinction between outer parameter draws and inner individual draws. This tutorial treats the chosen population model and its parameter uncertainty distribution as given. It does not include uncertainty about which structural model is correct.
The Variance Decomposition
For a prediction on the outcome scale, define \(m(\theta)=E(Y\mid\theta)\) and \(v(\theta)=\operatorname{Var}(Y\mid\theta)\). The predictive mean and variance are
The first variance component is the average within-population variance: expected variability among individuals conditional on the parameters. The second is the variance across parameter-dependent population means: uncertainty about the mean outcome implied by the model. Neither component alone generally describes the full predictive uncertainty for a future individual.
The decomposition follows directly by adding and subtracting the conditional mean. Write \(Y-E(Y)=[Y-m(\theta)]+[m(\theta)-E(Y)]\), square, and take expectations. The first squared term has expectation \(E_{\theta}\{\operatorname{Var}(Y\mid\theta)\}\). The second has expectation \(\operatorname{Var}_{\theta}\{m(\theta)\}\). The cross-product is zero because \(E\{Y-m(\theta)\mid\theta\}=0\), so its expectation conditional on \(\theta\) is zero. Adding the terms gives the stated result.
Worked Example: An Individual Clearance Prediction
Suppose, hypothetically, that human clearance \(Y\), in L/h, follows \(Y\mid\theta\sim N(\theta,3^2)\), where \(\theta\) is the population mean clearance. Let the uncertainty distribution for that mean be \(\theta\sim N(10,2^2)\). Assume the conditional population standard deviation of 3 L/h is known and fixed, and that the parameter distribution is normal as stated.
The predictive mean is \(E(Y)=E(\theta)=10\) L/h. The within-population variance is \(E\{\operatorname{Var}(Y\mid\theta)\}=3^2=9\ (\mathrm{L/h})^2\), while the parameter-uncertainty component is \(\operatorname{Var}(\theta)=2^2=4\ (\mathrm{L/h})^2\). Therefore,
Because both distributions are normal and the conditional variance is constant, the marginal predictive distribution is \(Y\sim N(10,13)\). Its central 90% predictive interval uses the normal multiplier 1.645:
Under the stated model, this interval is designed to cover 90% of outcomes for future individuals when uncertainty about \(\theta\) and population variability are both included. It is not an interval giving a 90% probability that the fixed population mean \(\theta\) lies within its endpoints. For comparison, omitting individual variability would leave only the uncertainty in \(\theta\), with standard deviation 2 L/h; that would address uncertainty about the population mean, not predict an individual clearance.
Implementing Nested Simulation
In a nested simulation, each outer draw fixes one plausible parameter set before the inner population outcomes are generated. For outer draw \(b\), write the parameter draw as \(\theta^{(b)}\), and generate inner outcomes \(Y^{(b,1)},\ldots,Y^{(b,K)}\) from \(p(Y\mid\theta^{(b)})\). Keeping the outer index is essential: it identifies which inner outcomes share the same parameter assumptions.
Generate \(B\) draws \(\theta^{(b)}\) from the specified parameter uncertainty distribution. Check that the draws respect parameter constraints and represent the intended uncertainty distribution.
For each \(\theta^{(b)}\), generate \(K\) individual outcomes from the population model \(p(Y\mid\theta^{(b)})\). Do not redraw the outer parameters independently for each inner individual when simulating a cohort under one common population model.
Retain individual outcomes for individual prediction, or calculate a cohort summary within each outer draw when the target is a future cohort. The target determines what is summarized.
Use the distribution of outcomes across outer draws, including their inner outcomes or conditional summaries as appropriate, to report predictive summaries and uncertainty components.
The number of inner replicates controls how accurately conditional population outcomes are represented for each parameter draw; the number of outer replicates controls how accurately parameter uncertainty is represented. Increasing one does not automatically solve inadequate simulation in the other. In practice, convergence should be assessed separately for quantities driven by the inner and outer layers.
Worked Example: Separating Uncertainty for an Individual Event
Consider a hypothetical binary event, such as an individual exceeding a prespecified exposure threshold. Suppose the uncertain human model has two equally likely parameter scenarios. Under the first, the event probability for an individual is \(q=0.20\); under the second, it is \(q=0.60\). Conditional on either scenario, individuals have independent Bernoulli outcomes with that scenario’s event probability.
The predictive event probability for one future individual is the average conditional probability:
The parameter-uncertainty contribution to the variance of the scenario-specific event probability is
For a Bernoulli outcome, the conditional variance is \(q(1-q)\). Its average across the two scenarios is \(\{0.20(0.80)+0.60(0.40)\}/2=0.200\). Thus the predictive variance of an individual’s event indicator is \(0.0400+0.200=0.240\). This equals \(0.40(1-0.40)\), as it must: after averaging over scenarios, the marginal outcome is Bernoulli with probability 0.40. Reporting only the variance of \(q\) would describe uncertainty in the event probability, not the full predictive variance of an individual event.
Predicting a Future Cohort
For a cohort of \(n\) independently sampled individuals under a common parameter set, let \(\bar{Y}_n\) be the cohort mean. Conditional on \(\theta\), its mean is \(m(\theta)\) and its variance is \(v(\theta)/n\). Applying the variance decomposition gives
The conditional individual variability is reduced by averaging across the cohort, but uncertainty about the population mean is not divided by \(n\): all individuals in the cohort share the same uncertain population parameters. This statement assumes conditional independence of individuals given \(\theta\). Dependence within a cohort would change the conditional variance term.
Worked Example: Predicting the Fraction With an Event
Continue with the two hypothetical scenarios above, but now consider the fraction with the event in a cohort of 100 individuals. Within each scenario, the event count is binomial with parameters \(n=100\) and \(q\). The conditional mean fraction is \(q\), and the conditional variance is \(q(1-q)/100\).
The parameter-uncertainty component is \(\operatorname{Var}(q)=0.0400\), as calculated above. The average within-cohort variance is
The predictive mean cohort fraction is 0.400, or 40.0%. Its predictive variance is \(0.0400+0.00200=0.0420\), and its predictive standard deviation is \(\sqrt{0.0420}=0.205\), or 20.5 percentage points. The large parameter component remains even with 100 individuals: the cohort may come from either of two distinct population scenarios. A calculation that treats all 100 outcomes as if they came from one fixed, known event probability would omit this component.
Worked Example: Retaining Precision in a Nested Variance Calculation
Suppose, hypothetically, an outer simulation of uncertain parameters and an inner simulation of a 100-person cohort produce a mean fraction at risk. Summarizing the full-precision outer results gives an estimated parameter component of \(0.0244382\) and an estimated average conditional cohort component of \(0.0022560\). These are variances on the fraction scale; the corresponding standard deviations are on the proportion scale.
The total predictive variance is found by adding the unrounded components:
Therefore, the predictive standard deviation is \(\sqrt{0.0266942}=0.1634\), or 16.34 percentage points. If the components were first rounded to \(0.02444\) and \(0.002256\), their displayed sum would be \(0.026696\); that rounded intermediate calculation should not replace the sum from the full-precision model output. Retaining precision until the final reported result avoids a small but preventable discrepancy.
Diagnostics, Alternatives, and Limitations
A nested prediction is only as trustworthy as its model, parameter uncertainty distribution, and simulation. Useful checks include comparing outer-draw summaries across independent simulation runs, increasing the number of outer draws and checking stability of predictive quantiles, and increasing inner replicates to assess the stability of conditional summaries. For a simulated cohort fraction, compare the simulated within-parameter variance with the theoretical binomial value \(q(1-q)/n\) when the conditional binomial assumption applies.
A common mistake is to pool every inner outcome and treat the resulting values as independent draws from a single fixed population. The inner outcomes within an outer draw share the same \(\theta\); they are conditionally generated under that parameter set, not under independently sampled parameter sets. Pooling can obscure how much predictions change across plausible population models. The correction is to retain the outer grouping, calculate conditional summaries within each group, and then summarize those results across parameter draws.
A plug-in prediction uses one parameter estimate and can describe conditional population variability, but it omits uncertainty across plausible parameter values. At the other extreme, an analytic calculation may integrate over parameters exactly and can be more efficient when the distributions allow it. Nested Monte Carlo is flexible when the prediction model is complex, but it introduces simulation error and requires enough draws at both levels. Neither simulation nor variance decomposition corrects a misspecified population model or an uncertainty distribution that fails to represent what is not known.
Predictive intervals should also be matched to their target. An interval for an individual outcome includes within-population variation and parameter uncertainty. An interval for a population mean concerns parameter uncertainty in that mean, while an interval for a finite future cohort includes both parameter uncertainty and residual cohort-to-cohort variation. State which quantity is predicted, the population and time horizon represented, and the units. A single interval without that context can invite an incorrect interpretation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A conditional individual outcome has variance 16, while the uncertain population mean has variance 9. What is the predictive variance for one individual, assuming these are the only two components?
- For a binary event with uncertain event probability \(q\), what does \(\operatorname{Var}(q)\) measure, and what additional term is needed for the variance of an individual event indicator?
- Why is parameter uncertainty not divided by cohort size when calculating the variance of a cohort mean under a common population parameter set?
- In a nested simulation, what should be increased if outer predictive quantiles remain unstable when inner conditional estimates are already stable?
Solutions. (1) The predictive variance is \(16+9=25\), so the predictive standard deviation is 5 in the outcome’s units. (2) \(\operatorname{Var}(q)\) measures uncertainty across parameter-dependent event probabilities; add \(E\{q(1-q)\}\) for the full predictive variance of the individual indicator. (3) Every individual shares the same uncertain population mean, so averaging reduces conditional individual variation but does not remove uncertainty about which mean applies. (4) Increase the number of outer parameter draws, because unstable outer quantiles indicate that parameter uncertainty is not yet represented precisely enough.