Separating Epistemic Uncertainty From Patient Variability

Two Sources of Spread, Two Different Questions

A QSP model can produce a broad range of predicted clinical outcomes for two fundamentally different reasons. The model may be uncertain about a biological parameter shared by all patients, such as a drug-effect coefficient. Even if that parameter were known, patients may differ in their physiology, baseline state, or other patient-level characteristics. These sources can both spread predictions, but they answer different questions and should not be treated as interchangeable.

The distinction matters when interpreting a virtual population. A prediction for one patient concerns both what is unknown about the shared model and how that patient differs from others. A prediction of the population distribution describes patient-to-patient differences under a specified model. A prediction of the population mean asks how uncertain that mean remains when shared model parameters are uncertain. Keeping those targets distinct prevents a model from turning uncertainty about one shared mechanism into artificial differences among patients.

This representation distinguishes the roles of the two inputs, not necessarily their scientific origins. For example, individual random effects may describe patient variability, while uncertain parameters governing the mean or spread of those effects are epistemic. A population distribution for \(U\) is treated as specified in the calculations below; uncertainty about that distribution would be another source of epistemic uncertainty.

The distinction also depends on the prediction target. Conditional on a fixed parameter value, the distribution of \(Y\) across \(U\) describes patient variability. Across plausible parameter values, the conditional population mean may itself vary. If the target is one future patient's outcome, both sources matter. If the target is the expected outcome in an effectively infinite population, patient variability determines the distribution of individual outcomes but does not by itself make that population mean uncertain once the parameters are fixed.

A Decomposition for a Clinical Prediction

Assume that \(Y=g(\boldsymbol{\Theta},U)\) has finite variance, that the parameter ensemble represents uncertainty about shared parameters, and that the distribution of \(U\) represents patient variability. Define the conditional population mean and conditional patient variance as

The conditional quantities describe a population under one fixed parameter draw. Averaging \(m(\boldsymbol{\Theta})\) across parameter draws gives the overall predictive mean. The variance of an individual prediction across both sources has the exact decomposition

The first term is the spread in conditional population means across plausible shared parameter values. The second is the average patient-to-patient variance within a fixed parameter draw. This identity separates the sources of spread for an individual prediction; it does not say that the two terms have the same scientific interpretation.

Why the Decomposition Holds

Write \(Y-E[Y]\) as the sum of a between-parameter component and a within-parameter component:

Squaring and taking expectations gives the variance of \(Y\). The cross-product has expectation zero because its conditional expectation given \(\boldsymbol{\Theta}\) is zero:

The remaining squared terms are \(\operatorname{Var}_{\boldsymbol{\Theta}}(m(\boldsymbol{\Theta}))\) and \(E_{\boldsymbol{\Theta}}[\operatorname{Var}(Y\mid\boldsymbol{\Theta})]\), which yields the stated result. The justification requires finite second moments and well-defined conditional distributions; it does not require a linear model or normally distributed predictions.

Individual Outcomes and Population Means

A population distribution and its mean are different prediction targets. With a fixed parameter draw, patient variability determines how dispersed individual outcomes are around the conditional mean. In an infinitely large population, that mean is \(m(\boldsymbol{\Theta})\). Across parameter draws, \(m(\boldsymbol{\Theta})\) can change, so the population mean can remain uncertain even when patient-to-patient variation averages away.

For a simulated cohort of \(N\) patients, the average outcome has an additional finite-cohort component. If patients are independent draws from the specified patient distribution conditional on the shared parameter, then

The first term does not shrink as more patients are simulated: drawing more patients under the same uncertain parameter does not reveal which shared parameter value is correct. The second term is the sampling variation of a finite cohort mean and decreases with \(N\) under the stated conditional independence assumption. This is a property of the simulated mean, not a claim that biological variability in individuals disappears.

Nonlinearity Can Make Patient Spread Depend on the Parameter

The two sources need not contribute additively in the model equation. If a parameter changes how strongly a patient characteristic affects the endpoint, then the patient variance can differ across parameter draws. The decomposition still applies, but the conditional patient variance \(v(\boldsymbol{\Theta})\) must be evaluated at each draw rather than assumed constant.

Simulation Practice, Diagnostics, and Limitations

A direct simulation should preserve the shared-versus-patient distinction. For each parameter draw, use that same draw for every patient in its simulated cohort; then draw patient-level characteristics according to the specified population distribution. Summarize outcomes within each parameter draw to inspect conditional population predictions, and compare those summaries across draws to inspect epistemic uncertainty. Pooling all simulated outcomes can be useful for an individual predictive distribution, but by itself it obscures which source produced the spread.

Two common alternatives answer narrower questions. A plug-in prediction fixes parameters at one selected value and varies patients, thereby describing patient variability conditional on that value but omitting parameter uncertainty. A parameter ensemble with no patient variation shows how the population prediction changes across plausible parameters but omits the distribution of individual outcomes. Neither is a complete substitute when the goal is to describe both sources. A nested simulation represents both, provided its sampling structure matches the clinical target.

Check the simulation design and summaries for these failure modes:

• Resampling the shared parameter for every patient. This makes patients appear to have different values of a parameter that is supposed to be shared. It converts epistemic uncertainty into artificial patient heterogeneity. Keep a parameter draw fixed across the cohort it represents.
• Calling the pooled spread “patient variability.” A pooled outcome distribution can contain both parameter uncertainty and patient differences. Compare within-draw patient distributions with the variation in their conditional means.
• Assuming a larger virtual cohort resolves every source of uncertainty. Conditional patient-sampling variation in a cohort mean can shrink with cohort size; uncertainty in a shared parameter does not shrink merely because more patients are simulated.
• Assuming patient variance is constant across parameter draws. This can fail when the model is nonlinear or when parameters change the response to patient characteristics. Evaluate the conditional spread at each draw.

These calculations are conditional on the chosen model, parameter ensemble, and patient distribution. They do not automatically include structural model uncertainty, uncertainty in the patient distribution itself, measurement error, or other sources not represented in \(Y=g(\boldsymbol{\Theta},U)\). Nor does the variance decomposition determine whether the ensemble or patient distribution is scientifically adequate; those require separate evaluation. The decomposition organizes represented uncertainty and variability rather than validating their inputs.