Model Uncertainty in Pharmacometric Simulations

1. What Is Model Uncertainty?

Model uncertainty arises when more than one plausible mathematical representation of a biological system could explain the available data. In pharmacometrics, this means that uncertainty may exist not only in the numerical values of model parameters, but also in the structure of the model itself.

For example, observed concentration-time data might be adequately described by either a one-compartment or two-compartment PK model over the sampling interval. A dose-response relationship might reasonably be represented by an E_max, sigmoid E_max, or indirect-response model. Each choice can produce different predictions when the model is used for simulation.

2. The Main Sources of Uncertainty

Several distinct sources of uncertainty can contribute to the distribution of simulated outcomes. Separating them conceptually is important because they require different approaches.

These components should not automatically be combined into one undifferentiated quantity. For example, between-subject variability represents heterogeneity among individuals, whereas uncertainty in the estimated population mean represents uncertainty about what that population parameter actually is.

3. Structural Model Uncertainty

Structural model uncertainty occurs when there are multiple scientifically plausible ways to describe the underlying PK, PD, disease, or treatment process.

Consider concentration-time data that can be described by either a one-compartment or two-compartment model. The two models may produce very similar predictions near the observed sampling times while diverging during extrapolation or under a different dosing regimen.

A two-compartment model instead contains additional distribution parameters and typically produces a concentration profile involving multiple kinetic time scales. The additional complexity may be scientifically useful, but it also introduces additional assumptions and parameters.

4. Parameter Uncertainty

Parameter uncertainty describes uncertainty about the numerical values of parameters after a model has been fitted.

Suppose a population PK model estimates clearance as:

The estimate of 5.0 L/h is not necessarily known exactly. Its uncertainty may be represented by a standard error, covariance matrix, confidence interval, or—in a Bayesian analysis—a posterior distribution.

For a vector of parameters \(\boldsymbol{\theta}\), frequentist simulation can represent approximate parameter uncertainty using:

where \(\hat{\boldsymbol{\theta}}\) is the estimated parameter vector and the variance-covariance matrix describes uncertainty and correlation among its components.

In Bayesian modeling, the corresponding concept is the posterior distribution:

Simulating across this distribution propagates uncertainty about the parameter values into uncertainty about future predictions.

5. Uncertainty Is Not the Same as Variability

One of the most important distinctions in pharmacometric simulation is between variability and uncertainty.

For example, a population PK model may specify:

where \(CL_{\text{pop}}\) describes the typical population value and \(\eta_i\) represents between-subject variability.

Uncertainty in \(CL_{\text{pop}}\) and variability in \(CL_i\) answer different questions. The former concerns how precisely the population parameter has been estimated; the latter concerns how much patients differ from one another.

6. Uncertainty in Covariate Relationships

Population PK/PD models frequently include covariates such as body weight, age, renal function, disease severity, or biomarker measurements.

A simple allometric relationship might be written as:

Here, uncertainty may exist in both the reference clearance \(CL_{\text{std}}\) and the estimated exponent \(\theta_{WT}\).

There may also be uncertainty about whether the relationship itself is appropriate. A linear, power-law, categorical, or spline relationship might each be scientifically plausible depending on the data and biological context.

Covariate uncertainty can therefore influence simulations in two ways: uncertainty in the parameters of the covariate relationship and uncertainty in the functional form of the relationship.

7. Residual Variability and Observation Error

Pharmacometric models commonly include an observation model describing unexplained variation between measured observations and model predictions.

For example, a proportional residual error model can be written as:

where \(F_{ij}\) is the model-predicted observation and \(\epsilon_{ij}\) represents residual variability.

When simulating future observations, residual variability can be important because the goal may be to predict what would actually be observed rather than merely the underlying typical trajectory.

However, residual error should not automatically be interpreted as biological variability. It can contain contributions from assay error, model misspecification, timing differences, unmodeled processes, and other sources.

8. How Uncertainty Propagates Into Predictions

Suppose the quantity of interest is a model prediction:

where \(\boldsymbol{\theta}\) represents model parameters and \(\mathbf{x}\) represents dosing or patient characteristics.

If the parameters are uncertain, then \(Y\) is also uncertain. Conceptually:

The prediction therefore reflects uncertainty about the parameters rather than treating the fitted parameter estimates as known constants.

This distinction becomes especially important when predictions are made far from the observations used to fit the model. Extrapolation can magnify differences between plausible models.

9. What Happens When Multiple Models Are Plausible?

Suppose three candidate models provide scientifically credible descriptions of the available data:

If all three models are plausible but lead to meaningfully different predictions, selecting one model and ignoring the others can understate model-related uncertainty.

Several strategies can be used:

• Scenario analysis: simulate each plausible model separately and compare the resulting predictions.
• Model averaging: combine predictions from multiple models using explicit model weights.
• Model uncertainty intervals: summarize prediction ranges across candidate models.
• Robust decision analysis: evaluate whether a decision remains acceptable across plausible models.

The appropriate approach depends on the purpose of the simulation and the evidence supporting the competing models.

10. Model Averaging

Model averaging combines predictions from multiple candidate models rather than conditioning inference entirely on one selected model.

If \(M_1,\ldots,M_K\) are candidate models with weights \(w_1,\ldots,w_K\), where:

then a model-averaged prediction can be expressed as:

where \(D\) denotes the available data.

The weights must be justified by an appropriate model-uncertainty framework. They should not simply be chosen to make the final prediction wider or narrower.

11. Worked Example: Propagating Parameter Uncertainty

Consider a hypothetical one-compartment IV model with a 500 mg dose. Suppose the fitted model gives:

• Typical clearance: \(CL=5.0\) L/h
• Volume of distribution: \(V=25\) L

The typical elimination rate constant is:

At 5 hours, the typical predicted concentration is:

Step 1: Introduce parameter uncertainty

Suppose uncertainty in clearance is represented by a distribution centered near 5 L/h, while volume is also uncertain. Instead of using exactly \(CL=5\) and \(V=25\) in every simulation, draw plausible parameter values from their joint uncertainty distribution.

Step 2: Recalculate the prediction

For each simulated parameter set:

Step 3: Repeat many times

After thousands of parameter draws, the simulations produce a distribution of predicted concentrations rather than one value.

Step 4: Summarize the prediction distribution

The resulting distribution can be summarized using a median or mean prediction together with an appropriate prediction or credible interval, depending on the simulation framework.

The important conceptual change is that the result is no longer simply:

Instead, the analysis recognizes that the model parameters themselves are uncertain and therefore the predicted concentration is uncertain.

12. Different Simulation Layers Represent Different Things

A useful pharmacometric simulation can contain several nested layers.

For example, an outer layer might sample uncertain population parameters, an inner layer might sample individual random effects, and another layer might generate residual observation variability.

Keeping these layers conceptually separate makes it easier to explain what a simulated interval actually represents.

13. Bootstrap Approaches to Model Uncertainty

Bootstrap methods can provide a practical way to assess uncertainty by repeatedly resampling subjects or observations, refitting the model, and repeating the simulation.

A simplified workflow is:

1. Sample subjects from the original dataset with replacement.
2. Refit the pharmacometric model to the bootstrap dataset.
3. Store the resulting parameter estimates.
4. Repeat the process many times.
5. Use the fitted models or parameter distributions to simulate the target scenario.

This approach can capture some of the uncertainty associated with finite sample size and model fitting without relying entirely on a local normal approximation to the parameter distribution.

14. Bayesian Simulation and Posterior Uncertainty

Bayesian pharmacometric models provide a natural framework for propagating parameter uncertainty because uncertainty is represented through posterior distributions.

After observing data \(D\), the posterior distribution is:

Simulation can then proceed by drawing parameter sets from the posterior and generating predictions under each draw.

This produces a posterior predictive distribution that incorporates uncertainty in the model parameters and, when specified, stochastic variation in future observations.

Bayesian analysis does not eliminate structural model uncertainty. If multiple structural models are plausible, model uncertainty still requires explicit consideration through model comparison, model averaging, sensitivity analysis, or another appropriate strategy.

15. Scenario Analysis for Model Uncertainty

Scenario analysis is often straightforward and transparent. Instead of combining competing assumptions into one distribution, simulate each plausible scenario separately.

For example:

Scenario analysis is especially useful when there is no compelling basis for assigning precise probabilities to competing structural assumptions.

It also provides a clear communication tool: rather than presenting one apparently precise prediction, the analyst can show how conclusions change under explicitly defined assumptions.

16. Sensitivity Analysis: Which Assumptions Matter?

Not every uncertainty source has the same effect on a simulation outcome. Sensitivity analysis examines how much the predicted outcome changes when important assumptions or inputs are varied.

Suppose the simulation target is exposure, \(AUC\). Under a simple linear IV model:

Uncertainty in clearance therefore directly influences the predicted exposure.

For a nonlinear PK or PK/PD model, the relationship may be much less obvious. Sensitivity analysis can identify whether the simulation is particularly dependent on:

• Clearance or volume parameters.
• Drug potency parameters such as EC₅₀.
• Between-subject variability.
• Covariate effects.
• Absorption assumptions.
• Turnover or disease-progression parameters.
• Competing structural models.
• Future population characteristics.

17. Why Extrapolation Can Magnify Model Uncertainty

Two models can fit the observed data similarly while making substantially different predictions outside the region where data were collected.

This is particularly relevant when simulations involve:

• Doses substantially higher or lower than those studied.
• Longer treatment durations.
• New patient populations.
• Unobserved disease states.
• Rare covariate combinations.
• Nonlinear processes outside the observed concentration range.

For example, a linear elimination model and a saturable elimination model may produce similar predictions at low concentrations but diverge at high concentrations.

When extrapolation is important, model uncertainty should therefore be considered explicitly rather than assuming that the best-fitting model is uniquely established by the observed data.

18. Using Model Uncertainty in Simulation-Based Decisions

Pharmacometric simulations are often used to support decisions about dose selection, trial design, sampling schedules, exposure targets, or development strategies.

When uncertainty is important, a useful question is not simply:

“What does the model predict?”

Instead, ask:

“How does the prediction change across the plausible uncertainty in the model and its parameters?”

For example, a dose may produce an acceptable exposure distribution under several plausible models but a substantially different result under one alternative structural assumption.

A simulation-based analysis can therefore report:

• Predictions under the primary model.
• Predictions under plausible alternative models.
• Parameter uncertainty around each model.
• Population variability among simulated individuals.
• Sensitivity of the target outcome to important assumptions.

This makes the simulation more transparent because the reader can distinguish what is driven by observed evidence from what depends on modeling assumptions.

19. A Practical Workflow for Model-Uncertainty Simulations

1. Define the simulation question. Specify the endpoint, population, dosing scenario, time horizon, and decision or scientific objective.
2. Identify the relevant uncertainty sources. Consider structural models, parameter estimates, covariate relationships, variability, and future scenario assumptions.
3. Establish the candidate model set. Include scientifically plausible alternatives rather than arbitrary models selected solely to widen the simulation range.
4. Evaluate model adequacy. Use diagnostic plots, predictive checks, parameter plausibility, convergence assessment, and relevant scientific knowledge.
5. Represent parameter uncertainty. Use an appropriate covariance-based, bootstrap, Bayesian, or other uncertainty representation.
6. Represent population variability separately. Distinguish uncertainty about population parameters from genuine between-subject heterogeneity.
7. Run simulations. Propagate the relevant uncertainty sources through the complete PK/PD or pharmacometric model.
8. Perform sensitivity analyses. Determine which assumptions have the largest effect on the target outcome.
9. Compare scenarios or models. Examine whether the scientific conclusion changes under plausible alternatives.
10. Communicate the uncertainty. State clearly which sources of uncertainty are represented and which remain outside the simulation.

20. Common Interpretation Mistakes

Model uncertainty is frequently misunderstood when several distinct concepts are combined.

• A prediction interval is not automatically a model-uncertainty interval. Its interpretation depends on exactly which sources of uncertainty were propagated.
• Between-subject variability is not parameter uncertainty. One represents population heterogeneity; the other represents uncertainty about estimated quantities.
• A good visual fit does not eliminate structural uncertainty. Multiple models can fit the same observations.
• More simulation replicates do not remove model uncertainty. Increasing the number of simulations reduces Monte Carlo error but does not make an uncertain model correct.
• Wider uncertainty is not automatically more realistic. Uncertainty distributions should be based on defensible assumptions and evidence.
• Model averaging is not the same as averaging arbitrary scenarios. Model weights require an explicit statistical or scientific justification.
• Extrapolation can be more uncertain than interpolation. Predictions outside the data-supported region may depend strongly on structural assumptions.

21. Monte Carlo Error Versus Model Uncertainty

Simulation itself introduces a separate numerical issue: Monte Carlo error.

Suppose the true simulation-based probability of an event is \(p\). If \(N\) independent simulation replicates are used, the estimated probability \(\hat p\) has approximate Monte Carlo standard error:

Increasing \(N\) makes the simulation estimate more stable.

But increasing \(N\) does not resolve uncertainty about the structural model or parameter values. It simply estimates the consequences of the specified simulation assumptions more precisely.

22. How to Report Model Uncertainty

A transparent pharmacometric simulation report should describe enough information for readers to understand what uncertainty was and was not propagated.

Clear reporting is particularly important when simulations are used to support clinical development decisions. The reader should be able to distinguish empirical evidence, model-based inference, and assumptions introduced for extrapolation.

23. A Complete Conceptual Simulation

Imagine a PK/PD model is being used to simulate the probability that a candidate dose achieves a target pharmacodynamic effect.

A comprehensive uncertainty simulation might proceed as follows:

1. Draw a plausible structural model from the candidate model set.
2. Draw population parameters from their uncertainty distribution.
3. Generate covariates for the simulated population.
4. Generate individual random effects representing between-subject variability.
5. Apply the dosing regimen.
6. Simulate the PK concentration-time profile.
7. Pass concentration into the PD model.
8. Generate the target effect or endpoint.
9. Repeat the process many times.
10. Repeat across alternative structural models or scenarios.

The resulting distribution can answer questions such as:

• What range of outcomes is predicted under the primary model?
• How much variation occurs among individuals?
• How much does parameter uncertainty contribute?
• How much do alternative structural models change the prediction?
• Does the target conclusion remain similar across plausible assumptions?

This is the central purpose of model-uncertainty analysis: not to manufacture uncertainty, but to reveal how uncertainty in the evidence and assumptions propagates into the quantities that matter.

Where to Go Next

A natural progression is to study model averaging in pharmacometric modeling, followed by Bayesian model averaging, parameter uncertainty propagation, bootstrap-based simulation, sensitivity analysis, and robust simulation strategies.

The next tutorial can build directly on these ideas by showing how competing PK/PD models can be combined through model averaging and how model weights influence simulated predictions.