1. What Is Model Averaging?
Model averaging is a framework for incorporating uncertainty about which candidate model best represents the data. Instead of selecting one structural model and treating that selection as fixed, model averaging combines information from several plausible models.
This distinction matters in pharmacometric modeling because different structural models can sometimes provide similar descriptions of the observed data while producing different predictions, parameter estimates, or extrapolations.
Model averaging retains information from multiple plausible models and combines them according to prespecified or estimated weights.
2. Why Is Model Uncertainty Important?
Pharmacometric analyses often involve structural decisions. Examples include choosing the number of compartments, the form of absorption, the representation of nonlinear elimination, or the structure of a covariate relationship.
These decisions are often informed by diagnostics, prior knowledge, plausibility, and statistical criteria. But when several models remain plausible, choosing one model can understate the uncertainty associated with that choice.
| Modeling decision | Candidate alternatives | Potential consequence |
|---|---|---|
| Number of compartments | One vs. two vs. three compartments | Different distribution and terminal-phase predictions |
| Absorption | First-order vs. transit vs. zero-order input | Different early concentration profiles and Tmax |
| Covariate form | Linear vs. power vs. nonlinear relationship | Different predictions for individuals outside the central covariate range |
| Residual error | Additive vs. proportional vs. combined | Different treatment of observation variability |
| PD relationship | Emax vs. sigmoid Emax | Different exposure-response predictions |
A model may fit the observed data well while alternative models produce similarly acceptable fits. The differences may become more important when the model is used for simulation, extrapolation, dose selection, or prediction in populations that differ from the development dataset.
3. Model Selection and Model Averaging Are Different
Model selection chooses one candidate model as the basis for subsequent inference. The selected model may be chosen using scientific considerations, diagnostics, likelihood-based criteria, information criteria, or a combination of evidence.
Model averaging retains several models and combines their results. The contribution of each model is determined by a model weight.
| Approach | Basic question | Output |
|---|---|---|
| Model selection | Which model should be used? | One selected model |
| Model averaging | How should information from plausible models be combined? | Weighted combination of models |
Model averaging is therefore not simply another model-selection criterion. It is a way of propagating structural model uncertainty into the quantity of interest.
4. Building a Set of Candidate Models
Model averaging begins with a candidate model set. The set should contain scientifically plausible alternatives rather than arbitrary models generated solely to increase the number of candidates.
For example, suppose a concentration-time dataset can plausibly be described by either a one-compartment or two-compartment disposition model. The candidate set might be:
where \(M_1\) is the one-compartment model and \(M_2\) is the two-compartment model.
For a more complex analysis, the candidate set could vary several structural features:
Each model is estimated using the same underlying dataset, although the models can contain different structural assumptions.
What belongs in the candidate set?
- Models that are scientifically plausible for the drug and study.
- Models that can be estimated with the available data.
- Models whose structural differences are relevant to the scientific question.
- Models that have been subjected to appropriate diagnostics.
The candidate set should be defined carefully because model averaging cannot compensate for a candidate set that excludes important plausible alternatives.
5. How Are Models Weighted?
Let \(w_k\) denote the weight assigned to candidate model \(M_k\). The weights typically satisfy:
A common approach is to derive weights from an information criterion. For example, if \(\mathrm{IC}_k\) is an information criterion for model \(k\), define:
A commonly used information-criterion weighting scheme has the form:
This expression converts relative information-criterion differences into normalized weights.
The exact interpretation of the weights depends on the criterion, candidate model set, and modeling framework. They should not automatically be interpreted as literal probabilities that a model is the “true” biological model.
6. AIC-Based Model Averaging
The Akaike information criterion (AIC) balances model fit against model complexity. In its common likelihood-based form:
where \(L\) is the maximized likelihood and \(p\) is the number of estimated parameters.
The first term rewards models that fit the data well, while the second term penalizes additional parameters.
After calculating AIC values for all candidate models, the differences are:
and the corresponding Akaike weights can be calculated as:
AIC weights are therefore relative weights within the specified candidate set. Adding or removing candidate models can change the normalized weights.
7. Other Ways to Construct Model Weights
AIC is not the only possible basis for model weighting. Other criteria may be useful depending on the inferential objective and modeling framework.
| Criterion or approach | General idea | Important consideration |
|---|---|---|
| AIC | Balances likelihood and parameter count using a relatively mild complexity penalty | Often used for predictive-oriented model comparison |
| AICc | Small-sample correction to AIC | Can be relevant when the effective sample size is not large relative to model complexity |
| BIC | Uses a stronger complexity penalty related to sample size | Has a different asymptotic motivation from AIC |
| Cross-validation | Evaluates predictive performance on held-out data | Requires an appropriate validation strategy |
| Bayesian model averaging | Weights models using posterior model probabilities | Depends on prior distributions and model evidence |
These approaches are not interchangeable. The weighting method should match the scientific objective, data structure, estimation framework, and assumptions of the analysis.
8. Averaging Predictions
One of the most direct applications of model averaging is to combine model predictions.
Suppose model \(M_k\) produces a prediction \(\hat{y}_k\) and receives weight \(w_k\). The model-averaged prediction is:
For a concentration prediction at time \(t\), this becomes:
This approach is especially natural when the scientific target is a prediction rather than a single parameter.
The averaged trajectory combines predictions from multiple candidate models. The resulting curve need not correspond to the trajectory generated by any single candidate model.
9. Can PK Parameters Be Averaged?
Parameters can sometimes be model-averaged, but this requires considerably more care than averaging predictions.
Suppose a parameter \(\theta\) has a directly comparable interpretation across all candidate models. A weighted model-averaged estimate can be written:
However, this calculation is meaningful only when \(\theta\) has a sufficiently common interpretation across the models.
For example, clearance may be directly comparable across several candidate structural models. In contrast, a peripheral volume in a two-compartment model has no direct counterpart in a one-compartment model.
| Quantity | Potentially comparable? | Reason |
|---|---|---|
| Systemic clearance | Often yes | May represent the same conceptual elimination quantity across models |
| Bioavailability | Often yes | Can have a common interpretation across candidate absorption models |
| One-compartment volume | Not necessarily | Its interpretation may differ from central or peripheral volumes in multi-compartment models |
| Peripheral volume | Often no | May exist only in models containing a peripheral compartment |
| Intercompartmental clearance | Often no | Only defined for models with the relevant distribution structure |
For this reason, prediction averaging is often easier to interpret than blindly averaging every parameter in the model.
10. Model Averaging and Uncertainty
One of the important motivations for model averaging is to recognize that uncertainty can arise from two different sources:
- Within-model uncertainty: uncertainty in parameter estimates conditional on a particular model.
- Between-model uncertainty: uncertainty associated with the choice among competing candidate models.
A conventional confidence interval or prediction interval based on one selected model primarily represents uncertainty conditional on that model.
Model averaging attempts to incorporate the additional uncertainty associated with the candidate model set.
Conceptually, if \(Y\) is a prediction and \(M\) denotes the model, the total predictive variance can be decomposed as:
The first term represents average within-model uncertainty, while the second represents variation among model-specific predictions.
11. Worked Example: Averaging a PK Prediction
Consider three hypothetical candidate PK models used to predict concentration at 12 hours after a dose. Suppose the models have the following information-criterion values and predictions.
| Model | Information criterion | 12-h prediction |
|---|---|---|
| Model 1 | 210 | 4.0 mg/L |
| Model 2 | 212 | 5.0 mg/L |
| Model 3 | 214 | 7.0 mg/L |
Step 1: Calculate the minimum criterion
Step 2: Calculate the differences
Step 3: Calculate the relative likelihood quantities
Step 4: Normalize the weights
The denominator is:
Therefore:
Step 5: Calculate the model-averaged prediction
The model-averaged prediction is therefore approximately 4.58 mg/L.
Notice that the averaged prediction is not identical to the prediction from any one of the three models. It incorporates the predictions of all three according to their relative weights.
12. Conditional and Model-Averaged Inference
Suppose a parameter or prediction is estimated conditional on model \(M_k\). The resulting quantity is model-specific:
Model averaging instead combines information over the candidate model set:
This distinction can be expressed conceptually as:
Model averaging therefore changes the inferential target. Instead of asking only what follows if one model is assumed, it incorporates uncertainty across the specified alternatives.
13. Model Averaging in Population Pharmacokinetics
In population PK, model uncertainty can occur at several levels. Structural uncertainty may involve compartment number, absorption model, covariate relationships, interindividual variability, or residual error structure.
For example, consider two candidate population models:
| Feature | Model A | Model B |
|---|---|---|
| Disposition | One compartment | Two compartments |
| Clearance covariate | Power relationship | Power relationship |
| Weight effect | Included | Included |
| Residual error | Proportional | Combined |
If both models remain plausible after diagnostic evaluation, model averaging can be used to combine predictions rather than relying exclusively on one structural specification.
Potential applications include:
- Population concentration prediction.
- Individual exposure prediction.
- Covariate-adjusted predictions.
- Simulation under alternative dosing regimens.
- Prediction in populations where structural uncertainty is consequential.
14. Averaging Alternative Covariate Models
Model uncertainty is particularly relevant when the goal is to predict drug exposure for patients with covariate values outside the center of the observed dataset.
Suppose clearance is modeled using body weight in two different ways:
versus a model with an alternative covariate structure:
These relationships can produce similar predictions near the center of the observed weight range but diverge at more extreme values.
Model averaging can combine the resulting predictions when both covariate structures are considered plausible.
15. Model Averaging in PK/PD Modeling
Model averaging can also be applied to pharmacodynamic models. Consider alternative exposure-response relationships such as a simple \(E_{\max}\) model and a sigmoid \(E_{\max}\) model.
A simple \(E_{\max}\) model is:
A sigmoid \(E_{\max}\) model adds a Hill coefficient:
If both models are plausible, model-specific predictions can be weighted and combined:
The same principle can be applied to indirect-response models, turnover models, disease-progression models, or other pharmacometric structures when multiple candidate models are scientifically plausible.
16. Model Averaging for Simulation
Simulation is often where structural uncertainty becomes especially important. A single selected model can generate a precise-looking simulated distribution even when several plausible models would produce different results.
There are several ways to incorporate model uncertainty into simulation. One conceptual approach is to:
- Define the candidate model set.
- Estimate each candidate model.
- Calculate model weights.
- Select a model according to its weight for each simulated replicate.
- Simulate from the selected model, including parameter and residual uncertainty as appropriate.
- Combine the resulting simulated distribution across replicates.
This approach differs from simply averaging the simulated concentration curves. Sampling a model according to its weight can preserve the fact that each model represents a different data-generating structure.
17. What Uncertainty Should Be Propagated?
Model averaging is only one component of uncertainty analysis. A pharmacometric prediction may contain several sources of uncertainty.
| Source | Example | Potential treatment |
|---|---|---|
| Parameter uncertainty | Uncertainty in CL or V | Covariance-based or simulation-based propagation |
| Interindividual variability | Between-subject variability in CL | Population simulation |
| Residual variability | Observation error | Residual-error simulation |
| Covariate uncertainty | Unknown future patient characteristics | Appropriate covariate distributions or scenarios |
| Structural model uncertainty | One vs. two compartments | Model averaging or model-selection sensitivity analysis |
A robust simulation strategy should identify which uncertainty sources are relevant to the decision rather than treating model averaging as a universal substitute for uncertainty propagation.
18. Limitations and Practical Considerations
Model averaging can be useful, but it does not eliminate the need for careful pharmacometric reasoning.
- The candidate model set matters. Important plausible models omitted from the set cannot contribute to the averaged result.
- Weights depend on the weighting method. AIC, AICc, BIC, cross-validation, and Bayesian approaches can produce different weights.
- Model weights depend on the candidate set. Adding or removing models can change normalized weights.
- Parameter averaging requires common interpretation. Parameters that exist only in some models should not automatically be averaged.
- Averaged predictions need interpretation. The averaged curve may not correspond to the output of any single mechanistic model.
- Model averaging does not correct structural misspecification automatically. If all candidate models are poor, averaging them does not create a valid model.
- Complexity can increase. Maintaining, validating, and communicating multiple models requires additional work.
- Decision relevance matters. Model uncertainty should receive particular attention when competing models produce materially different conclusions for the scientific question.
19. A Practical Model-Averaging Workflow
- Define the scientific question. Decide whether the target is a parameter, concentration prediction, exposure measure, simulation outcome, or another quantity.
- Identify structural uncertainties. Determine which modeling choices are genuinely uncertain and scientifically relevant.
- Construct the candidate model set. Include plausible alternatives rather than arbitrary variations.
- Estimate every candidate model. Use a consistent dataset and appropriate estimation procedures.
- Evaluate model adequacy. Examine diagnostics, parameter plausibility, identifiability, and predictive performance.
- Select a weighting framework. For example, information-criterion weights or a Bayesian model-averaging framework.
- Calculate model weights. Confirm that the weights are nonnegative and sum to one.
- Generate model-specific quantities of interest. These may include concentrations, AUC, exposure-response predictions, or simulations.
- Combine the results. Average predictions or otherwise integrate across models according to the chosen framework.
- Propagate uncertainty. Include parameter uncertainty and other relevant sources in addition to model uncertainty.
- Perform sensitivity analyses. Assess how conclusions change under alternative candidate sets or weighting procedures when appropriate.
- Communicate the model set and weights. Report enough information for the reader to understand how the averaged result was constructed.
20. How Should Model Averaging Be Reported?
A model-averaging analysis should make the source of the averaged result transparent.
At minimum, reporting should describe:
- The candidate models considered.
- The scientific rationale for including those models.
- The estimation method used for each model.
- The model-comparison criterion or weighting framework.
- The resulting model weights.
- The quantity that was averaged.
- How parameter and predictive uncertainty were propagated.
- Any sensitivity analyses involving the candidate model set.
For example, a concise report might state that three prespecified structural PK models were evaluated, AIC-based weights were calculated, and concentration predictions were averaged using those weights.
21. What Does a Model-Averaged Result Mean?
A model-averaged result should be interpreted as an inference or prediction that integrates across a defined set of candidate models according to a defined weighting scheme.
It does not mean that the drug simultaneously follows several compartment structures in a literal biological sense.
For example, if a model-averaged concentration is:
the appropriate interpretation is that 5.2 mg/L is the weighted combination of the model-specific predictions under the specified candidate set and weights.
The interpretation therefore depends on three things:
- Which models were included?
- How were they weighted?
- What quantity was averaged?
22. Model Selection Versus Model Averaging
| Feature | Model selection | Model averaging |
|---|---|---|
| Number of models used for final inference | Usually one | Multiple |
| Structural uncertainty | Conditioned on selected model | Explicitly incorporated across candidate models |
| Output | Model-specific estimate or prediction | Weighted combination |
| Interpretability | Usually straightforward for model parameters | Requires explanation of candidate models and weights |
| Parameter averaging | Direct within selected model | Requires parameters to have comparable meanings |
| Prediction averaging | Prediction from selected model | Weighted prediction across models |
| Computational burden | Lower after model selection | Higher because multiple models must be estimated and evaluated |
Neither framework removes the need for scientific judgment. The important distinction is whether uncertainty about the structural model is conditioned away through selection or retained as part of the analysis.
23. Implementing Model Averaging in Pharmacometric Software
Model averaging is not limited to a particular software package. The general workflow can be implemented whenever the analyst can estimate multiple candidate models, calculate appropriate weights, generate model-specific predictions, and combine the results.
A practical implementation can be organized into four computational layers:
| Layer | Task |
|---|---|
| Model fitting | Estimate each candidate pharmacometric model |
| Model comparison | Calculate likelihood-based or predictive criteria |
| Weight calculation | Convert relative model performance into normalized weights |
| Prediction combination | Generate and combine model-specific predictions or simulations |
For reproducibility, it is useful to maintain a structured record containing each model's specification, parameter estimates, objective function or likelihood, information criterion, weight, and predictions.
This is particularly important when model averaging is performed across a large set of candidate population PK or PK/PD models.
24. When Does Model Averaging Matter Most?
Model uncertainty becomes particularly important when candidate models make similar predictions within the observed data but diverge for the intended application.
Examples include:
- Predicting concentrations at sampling times not represented in the original dataset.
- Simulating alternative dosing regimens.
- Extrapolating to different patient populations.
- Predicting exposure at extreme covariate values.
- Evaluating long-term disease or treatment trajectories.
- Using a PK/PD model to predict effects at concentrations beyond those directly observed.
If all plausible models lead to essentially the same decision-relevant prediction, structural model uncertainty may have limited practical impact. If predictions diverge substantially, explicitly accounting for model uncertainty can become much more consequential.
25. Key Takeaways
- Model averaging incorporates uncertainty across multiple plausible pharmacometric models rather than conditioning all inference on one selected model.
- A candidate model set should contain scientifically plausible and adequately evaluated alternatives.
- Model weights quantify each candidate model's contribution under the chosen weighting framework.
- A common information-criterion approach uses differences such as \(\Delta_k=IC_k-IC_{\min}\) to construct normalized weights.
- AIC-based weights are relative to the specified candidate model set and should not automatically be interpreted as probabilities that models are biologically true.
- Model-averaged predictions can be calculated as a weighted combination of model-specific predictions.
- Prediction averaging is generally easier to interpret than parameter averaging when candidate models have different parameterizations.
- Parameters should only be averaged when they have sufficiently comparable meanings across candidate models.
- Model averaging can incorporate between-model uncertainty in addition to within-model parameter uncertainty.
- Simulation can incorporate model uncertainty either by averaging predictions or by sampling from candidate models according to their weights.
- Model averaging does not compensate for a poor candidate model set or structural misspecification shared by all candidate models.
- The weighting method, candidate model set, and quantity being averaged should be clearly reported.
- Model uncertainty is especially relevant when plausible models produce materially different predictions for the intended scientific application.
Where to Go Next
A natural progression is to study population PK model selection and validation, followed by covariate model building, bootstrap and visual predictive checks, Bayesian model averaging, and simulation-based evaluation of structural model uncertainty.
The next tutorial can build directly on this topic by examining population PK model selection in detail, including structural model development, covariate selection, diagnostic evaluation, and the distinction between statistical criteria and pharmacometric plausibility.