Why the Source of Uncertainty Matters
An integrated prediction can be uncertain for several fundamentally different reasons. A future patient may differ from the average patient; a measured outcome may depart from its expected value because of residual error; and the model parameters may not be precisely known. These sources can all widen a predictive distribution, but they do not describe the same thing and should not be interpreted or simulated as if they did.
The distinction matters when a prediction is used for a drug development decision. Between-subject variability describes the range of outcomes expected across people under the modeled population. Parameter uncertainty describes how much the prediction could change as knowledge about the model improves. Residual error describes remaining variation in an observation after accounting for the modeled predictors and individual effects. A prediction for a future patient's measured outcome may include all three. A prediction of the population mean, or of a latent outcome without measurement error, may include fewer.
The introductory material on nested simulation provides the general strategy of drawing uncertain quantities at their appropriate levels. Here the emphasis is on deciding what each level represents and what belongs in a particular predictive distribution.
A Hierarchical View of an Integrated Prediction
Let \(D\) denote the evidence used to fit the integrated model, and let \(\theta\) denote its parameters, including any estimated population or residual variance parameters. In a Bayesian analysis, parameter uncertainty is represented by the posterior distribution \(p(\theta\mid D)\). For an individual \(i\), let \(\eta_i\) represent between-subject effects, and let \(\epsilon_{ij}\) represent residual error for observation \(j\) on that individual. A simple observation model is
Here \(g\) may combine linked model components, such as an exposure prediction and an exposure-response function. The exact form of \(g\) depends on the scientific model. The roles of the quantities are more general:
- Parameter uncertainty: uncertainty about the values of \(\theta\), conditional on the evidence and the model.
- Between-subject variability: variation in \(\eta_i\) across individuals, conditional on the population model and its parameters.
- Residual error: variation in \(\epsilon_{ij}\) around the model's conditional expectation, for example observation error or remaining within-individual variation.
A common set of assumptions makes the decomposition especially clear: residual error has conditional mean zero; conditional on \(\theta\) and \(\eta_i\), it has variance \(\sigma^2(\theta,\eta_i)\); and it is independent across observations when the model says observations have independent residuals. The individual effect \(\eta_i\) is drawn from its modeled population distribution given \(\theta\). Parameter draws, individual effects, and residual draws must follow the model's actual conditional dependencies. Independence is not automatic: for example, random effects can be correlated, and their distribution may depend on parameters.
How Predictive Variance Separates Into Sources
For a future quantity \(Y\), the law of total variance separates variation conditional on the parameters from variation in the conditional prediction as the parameters vary:
The first term is the average conditional predictive variance. For a future individual observation, it can include between-subject variability and residual error. The second term is the variance, across plausible parameter values, of the conditional mean prediction. It represents parameter uncertainty's contribution to predictive variance. If the prediction target is a latent response rather than a measured observation, residual error is omitted; if it is a population mean rather than an individual outcome, the individual random effect may be averaged out instead of included as a realized effect.
For the additive model \(Y=g(\theta,\eta)+\epsilon\), suppose \(E(\epsilon\mid\theta,\eta)=0\), the residual is conditionally independent of \(\eta\), and its conditional variance is \(\sigma^2(\theta,\eta)\). Conditional on \(\theta\), the variance can then be written as
The first term is the contribution from between-subject effects; the second is the average residual contribution. Averaging this conditional variance over the parameter distribution and adding the variance of the conditional mean gives the full predictive variance. These terms can be added as separate variances only under the stated conditional structure. If components are dependent, their covariance contributions must also be represented.
Worked Example: Three Sources in a Simple Future-Observation Model
Suppose a hypothetical model for a future measurement, in milligrams per liter, is \(Y=\theta+\eta+\epsilon\). Based on the available evidence, let \(\theta\) have a normal posterior distribution with mean \(10\) mg/L and standard deviation \(1.5\) mg/L. For a new individual, let \(\eta\) be normally distributed with mean zero and standard deviation \(2\) mg/L. Let the residual error \(\epsilon\) be normally distributed with mean zero and standard deviation \(1\) mg/L. Assume the three terms are independent. Find the predictive mean, variance, and an approximate 95% predictive interval.
The predictive mean is \(10+0+0=10\) mg/L. Since the independent terms are additive, their variances contribute separately:
The predictive standard deviation is \(\sqrt{7.25}=2.693\) mg/L. Under the normal assumptions in this example, an approximate 95% predictive interval is
The interval describes the approximate range for a future measured value under this model, not a 95% probability that the fixed parameter \(\theta\) lies in this particular interval. The parameter variance contributes \(2.25\;(\mathrm{mg/L})^2\); between-subject variability contributes \(4\;(\mathrm{mg/L})^2\); and residual error contributes \(1\;(\mathrm{mg/L})^2\).
Following Uncertainty Through a Model Handoff
In an integrated model, the output of one component may become an input to another. Its uncertainty should be carried forward with its meaning intact. For example, uncertainty in a population PK parameter is not itself a new patient's PK random effect. Nor is a predicted exposure's residual error automatically the same as residual error in a downstream clinical outcome. A downstream model should receive the upstream quantity and uncertainty appropriate to its interface, without silently changing the source or counting it twice.
Worked Example: A Log-Scale Exposure Prediction
Consider a hypothetical observed exposure \(A\), in mg·h/L, modeled on the log scale as \(\log(A)=m+\eta+\epsilon\). The uncertain population log-location \(m\) has posterior mean \(\log(12)\) and posterior standard deviation \(0.10\). For a new individual, \(\eta\) has mean zero and standard deviation \(0.25\). Residual log-scale error \(\epsilon\) has mean zero and standard deviation \(0.10\). Assume independent normal terms. Calculate the median and an approximate 95% predictive interval for observed exposure.
Because the terms on the log scale are independent and normal, \(\log(A)\) is normal with mean \(\log(12)\). Its variance includes parameter uncertainty, between-subject variability, and residual error:
Thus the log-scale standard deviation is \(\sqrt{0.0825}=0.2872\). Exponentiating the predictive median gives \(12\) mg·h/L. Exponentiating the normal 95% interval on the log scale gives
This is a predictive interval for observed exposure under the assumed model. It is asymmetric on the original scale because the model is normal on the log scale. If the target were a typical individual's latent exposure, residual error would be left out; if the target were the population-average exposure, the random-effect averaging would need to be handled for that target rather than interpreted as a prediction for a randomly selected individual.
Worked Example: Combining an Upstream Prediction With a Downstream Outcome
Suppose a hypothetical integrated model predicts an upstream exposure-like quantity \(X\) and then an outcome \(Y\), both in arbitrary units. Let \(X=\theta+\eta_X\), where \(\theta\) has posterior mean \(8\) and standard deviation \(1\), and the between-subject effect \(\eta_X\) has mean zero and standard deviation \(2\). Let \(Y=0.5X+\eta_Y+\epsilon\), where the downstream between-subject effect \(\eta_Y\) has standard deviation \(1.5\), and residual error \(\epsilon\) has standard deviation \(1\). Assume all random terms are independent and centered as specified. Find the predictive mean and variance of \(Y\) for a future individual observation.
Substituting the upstream expression gives \(Y=0.5\theta+0.5\eta_X+\eta_Y+\epsilon\). The mean is \(0.5(8)=4\) units. The parameter uncertainty contributes \(0.5^2(1^2)=0.25\) units squared. Upstream between-subject variability contributes \(0.5^2(2^2)=1\) unit squared, while downstream between-subject variability contributes \(1.5^2=2.25\) units squared. Residual error contributes \(1^2=1\) unit squared. Therefore,
The approximate normal 95% predictive interval is \(4\pm1.96(2.121)=[-0.16,\;8.16]\) units. A negative lower limit may be scientifically implausible if the real outcome cannot be negative; that signals a limitation of this illustrative linear-normal model, not a reason to relabel or remove one of the variance sources. A more suitable outcome distribution may be required for such an application.
Diagnostics, Alternatives, and Limitations
A source-by-source audit is useful whenever an integrated predictive distribution looks unexpectedly narrow or wide. First specify the target: a population mean, a latent outcome for a new individual, or a future measured observation. Then check that the random quantities included match that target. For a new person's repeated measurements, the person's \(\eta_i\) is typically drawn once and shared across that person's occasions; independently redrawing it at every occasion would represent different individuals and distort within-person dependence. Residual errors may be drawn separately by occasion only if the residual model supports that assumption.
Diagnostics should also examine whether the model's implied variation is plausible at each level. Compare predicted between-person spread with the observed spread across people, and examine within-person residual patterns where repeated measurements are available. Check posterior uncertainty in parameters that control the mean and variance, not just uncertainty in the mean prediction. In an integrated chain, verify that a quantity passed across a model interface has not already had a source of error added that the receiving component adds again. When the model implies dependence among random effects or parameters, independent draws can give the wrong predictive distribution even if each marginal distribution looks reasonable.
| Approach | What it represents | Main limitation |
|---|---|---|
| Plug-in prediction at one parameter estimate | Conditional prediction for fixed parameter values | Does not propagate uncertainty in those parameters. |
| Predictive distribution integrating parameter draws | Uncertainty in parameters plus the modeled random variation for the target | Depends on a well-specified model and a correct target-specific simulation. |
| Population-mean prediction | Expected outcome across the modeled population | Is not a prediction interval for an individual patient. |
The plug-in approach can be appropriate when the goal is specifically a conditional prediction at fixed parameter values, but it is usually too narrow for a predictive distribution intended to reflect uncertainty about a future outcome. Conversely, adding every available variance term is not automatically conservative or correct: some terms may not belong to the prediction target, may be dependent, or may already be represented elsewhere. The decomposition also relies on the model's assumptions. It cannot establish that the distributions, covariate relationships, population, or measurement process are adequate merely because the calculation is correct.
Common pitfall: “All three standard deviations describe uncertainty, so I can combine them without saying what they mean.” This is wrong because the terms refer to different levels and prediction targets. The correction is to write the generative model, identify whether the target is a population mean, an individual latent outcome, or an observed outcome, and include each source once according to that model. In particular, parameter uncertainty is not between-subject variability, and residual error is not a substitute for either.
State whether the prediction is for a population summary, a future individual, or a future observation.
Show where parameters, individual effects, and residual terms enter, including any shared effects across linked components or repeated observations.
Integrate parameter uncertainty and draw only the random effects and residual terms needed for the stated target, preserving modeled dependence.
Inspect source-specific contributions, units, and plausibility; compare them with available between- and within-subject information.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A model predicts a population-average latent response. Should residual error for a future assay be included in that prediction? Explain.
- In \(Y=\theta+\eta+\epsilon\), the independent standard deviations are \(0.5\), \(1\), and \(2\) units, respectively. What is the predictive variance for a future measured outcome, and which term is largest?
- For repeated measurements on one future individual, should a subject random effect generally be redrawn independently at each occasion?
- Why can a plug-in prediction at the posterior mean parameter be narrower than a predictive distribution that integrates over parameter uncertainty?
Solutions. (1) No. A population-average latent response is not a future measured value; include assay residual error only when predicting an observation for which that error is part of the target. (2) The variance is \(0.5^2+1^2+2^2=0.25+1+4=5.25\) units squared. The residual term is largest, contributing \(4\) of the \(5.25\) units squared. (3) Generally no. The same individual's random effect is shared across that individual's occasions under a model with a persistent subject effect; drawing it again each time changes the implied dependence. (4) A plug-in prediction holds the parameter fixed and omits variation in the conditional prediction across plausible parameter values. Integrating over parameter uncertainty adds that contribution to the predictive distribution's variance.