Bayesian Model Averaging Across Mechanisms

Why Average Across Mechanisms?

A mechanistic comparison may leave substantial support for more than one biological explanation. For example, after fitting models for pathway feedback and feedforward control, the data may favor one mechanism without eliminating the other. If the purpose is to predict an endpoint under a dose or perturbation, reporting only the prediction from the most supported mechanism hides the remaining uncertainty about which mechanism generated the data.

Bayesian model averaging (BMA) combines predictions using the posterior probabilities of the candidate models. It does not assert that the mechanisms are literally interchangeable or that all are true. Rather, conditional on the specified candidate set, it represents uncertainty about which model is appropriate by retaining each model's predictive distribution with a data-updated weight. The result depends on the candidate models, their prior probabilities, and the evidence used to update those probabilities.

Definition: Let $M_1,\ldots,M_K$ be candidate mechanisms, $D$ the observed data, and $\tilde Y$ a future or otherwise unobserved QSP endpoint under a specified intervention. Bayesian model averaging defines the predictive distribution as $$ p(\tilde Y\mid D)=\sum_{k=1}^{K}P(M_k\mid D)\,p(\tilde Y\mid D,M_k). $$ Here $P(M_k\mid D)$ is the posterior probability of mechanism $M_k$, and $p(\tilde Y\mid D,M_k)$ is that mechanism's posterior predictive distribution. The model probabilities are nonnegative and sum to one.

For the formula to answer a meaningful scientific question, the models must predict the same endpoint under the same intervention and relevant observation definition. Their likelihoods must be based on comparable evidence, and the model probabilities must be based on stated prior model probabilities and marginal likelihoods. Within each model, the predictive distribution should account for the relevant uncertainty in that model's parameters and observations. Fair comparison of evidence relies on shared comparison assumptions, as discussed in the earlier material on shared assumptions for fair model comparisons.

From Evidence to Model Weights

For each candidate model, Bayesian updating combines its prior probability with the probability of the observed data under that model. That latter quantity is the marginal likelihood: it averages the likelihood over the model's parameter prior. For model $M_k$, write it as $p(D\mid M_k)$. The resulting model probability is

P(M_k\mid D)= \frac{p(D\mid M_k)P(M_k)} {\sum_{j=1}^{K}p(D\mid M_j)P(M_j)}.

The posterior probability is a weight for prediction, not a parameter estimate within the mechanism. Marginal likelihood ratios can be expressed as Bayes factors; a Bayes factor compares the data support for two models, while the posterior odds also depend on their prior odds. A model with the largest posterior probability is the leading candidate in this set, but its probability need not be close to one.

Worked Example: Updating Probabilities for Three Mechanisms

Suppose three hypothetical mechanisms have prior probabilities $P(M_1)=0.50$, $P(M_2)=0.30$, and $P(M_3)=0.20$. For the observed data $D$, suppose their marginal likelihoods relative to $M_1$ are $1$, $2$, and $0.5$, respectively. These are relative evidence values, not posterior probabilities.

The unnormalized posterior weights are prior probability times relative marginal likelihood:

(0.50)(1)=0.50,\qquad (0.30)(2)=0.60,\qquad (0.20)(0.5)=0.10.

Their sum is $0.50+0.60+0.10=1.20$, so the normalized model probabilities are

P(M_1\mid D)=\frac{0.50}{1.20}=0.4167,\quad P(M_2\mid D)=\frac{0.60}{1.20}=0.5000,\quad P(M_3\mid D)=\frac{0.10}{1.20}=0.08333.

Thus $M_2$ has the greatest posterior probability, but the other two mechanisms together still have posterior probability $0.5000$. The weights sum to one, as required. These probabilities are conditional on this candidate set, the stated prior probabilities, and the observed data.

Averaging Predictions and Their Uncertainty

Each candidate model produces a predictive distribution, not just a point prediction. If the endpoint has a finite mean under each model, let $\mu_k=E(\tilde Y\mid D,M_k)$ and $\sigma_k^2=\operatorname{Var}(\tilde Y\mid D,M_k)$. The model-averaged predictive mean is the weighted mean of the model-specific predictive means. The predictive variance includes both uncertainty within each model and dispersion among the model-specific means.

E(\tilde Y\mid D)=\sum_{k=1}^{K}w_k\mu_k,\qquad \operatorname{Var}(\tilde Y\mid D) =\sum_{k=1}^{K}w_k\left[\sigma_k^2+(\mu_k-\bar\mu)^2\right],

where $w_k=P(M_k\mid D)$ and $\bar\mu=\sum_{k=1}^{K}w_k\mu_k$. The term $\sigma_k^2$ is within-model predictive variance; the squared deviation $(\mu_k-\bar\mu)^2$ contributes between-model variance. Both matter: models may be individually precise but disagree with one another, or agree on their means while each remains uncertain.

Justification: Treat the model identity $M$ as a random variable with $P(M=M_k\mid D)=w_k$. Conditional expectation gives $E(\tilde Y\mid D)=\sum_k w_k\mu_k=\bar\mu$. For the variance, condition on each model and expand: $$ E\!\left[(\tilde Y-\bar\mu)^2\mid D,M_k\right] =E\!\left[((\tilde Y-\mu_k)+(\mu_k-\bar\mu))^2\mid D,M_k\right]. $$ The cross-product has expectation zero because $E(\tilde Y-\mu_k\mid D,M_k)=0$. The remaining terms are $\sigma_k^2+(\mu_k-\bar\mu)^2$. Averaging these conditional expectations with weights $w_k$ gives the stated variance formula.

Worked Example: Averaging a QSP Endpoint and Its Variance

Consider a hypothetical prediction of target suppression at 24 hours, measured in percentage points, under one specified dose. Suppose three mechanisms have posterior probabilities $0.60$, $0.30$, and $0.10$. Their posterior predictive means are $42$, $55$, and $35$ percentage points, and their posterior predictive standard deviations are $6$, $8$, and $5$ percentage points. These predictive summaries are assumed to include parameter and observation uncertainty relevant to the endpoint.

The model-averaged mean suppression is

\bar\mu=(0.60)(42)+(0.30)(55)+(0.10)(35) =25.2+16.5+3.5=45.2\text{ percentage points}.

First calculate within-model variance, using the squared standard deviations:

(0.60)(6^2)+(0.30)(8^2)+(0.10)(5^2) =21.6+19.2+2.5=43.3\text{ percentage-points}^2.

The between-model component is

(0.60)(42-45.2)^2+(0.30)(55-45.2)^2+(0.10)(35-45.2)^2 =6.144+28.812+10.404=45.36\text{ percentage-points}^2.

Therefore the total predictive variance is $43.3+45.36=88.66\text{ percentage-points}^2$, and its square root is $\sqrt{88.66}=9.416$ percentage points. The between-model component is substantial: disagreement among mechanisms contributes slightly more variance than uncertainty within them. The mean of 45.2 percentage points is a weighted prediction, not a claim that any single mechanism predicts exactly that value. Nor does the mean and standard deviation alone determine a valid interval; the mixture may be skewed or multimodal.

Average the Predictive Quantity of Interest

BMA applies to the full predictive distribution. Consequently, the correct calculation depends on the quantity being asked for. For an event such as suppression exceeding a threshold, average the model-specific event probabilities. Do not first average endpoint means and then treat the result as if it came from one model; nonlinear transformations and thresholding generally do not commute with averaging.

Worked Example: Averaging the Probability of Crossing a Threshold

Suppose the hypothetical posterior model probabilities are $0.50$, $0.30$, and $0.20$. Under the same intervention, the three mechanisms assign probabilities $0.70$, $0.40$, and $0.20$ to achieving at least 50% target suppression at 24 hours. The model-averaged probability of that event is

P(\tilde Y\geq 50\mid D) =(0.50)(0.70)+(0.30)(0.40)+(0.20)(0.20) =0.35+0.12+0.04=0.51.

The averaged event probability is $0.51$, or 51%. Choosing $M_1$ alone would report 70%, but that ignores the posterior probability assigned to the other mechanisms. The answer concerns the specified endpoint, threshold, intervention, and observed data; it is not a probability that a mechanism is true.

Model Averaging Versus Selecting One Model

Approach	Prediction	What it retains	Main limitation
Select the highest-probability model	Use the prediction from the model with largest $P(M_k\mid D)$	Parameter and predictive uncertainty within that chosen model, if calculated	Discards uncertainty about which candidate mechanism is appropriate
Bayesian model averaging	Combine model-specific predictive distributions using $P(M_k\mid D)$	Within-model uncertainty and disagreement across candidate mechanisms	Depends on the candidate set, model priors, and comparability of evidence

Selection can be useful when a single mechanism is needed for a specific scientific or operational purpose, but a leading model is not automatically a certain model. Averaging is appropriate when the inferential target is a prediction that should account for uncertainty among the stated alternatives. It can also reveal that the prediction is robust despite model uncertainty: if highly weighted models make similar predictions, their mixture may be close to each individual prediction. Conversely, strong disagreement can make a single selected prediction misleading.

Diagnostics, Assumptions, and Common Pitfalls

Before interpreting an averaged prediction, check that the weights and predictions answer the same question. In practice, examine the following:

Weight calculation: Confirm that posterior model probabilities are normalized, use the intended prior model probabilities, and are based on marginal likelihoods for the same observed evidence.
Endpoint alignment: Confirm that every model predicts the same endpoint, time point, population or system, dose, and intervention. A weighted average of unlike quantities has no clear interpretation.
Predictive distributions: Check that each model's distribution includes the uncertainties relevant to the prediction, rather than only a fitted deterministic trajectory or parameter estimate.
Model-set sensitivity: Recognize that posterior probabilities are conditional on the included mechanisms. A missing plausible mechanism is not represented by the average.
Prediction disagreement: Inspect the component predictions and the between-model variance, not only the averaged mean. A single summary can conceal separated modes or a wide range of plausible outcomes.
Prior sensitivity: Since marginal likelihoods average over parameter priors, model probabilities can change with prior choices. Check whether plausible prior specifications materially alter the weights or predictions.

A frequent mistake is: “$M_1$ has the largest posterior probability, so its prediction is the Bayesian prediction.” That is model selection, not model averaging. The correction is to combine the predictive distributions using all posterior model probabilities, unless the scientific question explicitly calls for a selected mechanism. Another mistake is to average model-specific standard deviations directly. The variance formula shows why this fails: variances are combined with both within-model and between-model terms, and standard deviations are not additive in that way.

BMA does not repair misspecification shared by all candidates, guarantee that the candidate set includes the true biological process, or make incompatible models comparable. It is also not a substitute for checking whether the mechanistic alternatives reproduce relevant data patterns. The earlier material on posterior predictive checks addresses model-data discrepancies; averaging predictions cannot make a systematic failure disappear. Finally, an averaged point prediction is not itself a mechanistic explanation: report the component mechanisms and their weights when biological interpretation matters.

Key takeaway: Average model-specific predictive distributions with posterior model probabilities. For a mean, this gives a weighted mean; for uncertainty, retain both within-model variance and between-model disagreement. Selecting the most probable mechanism is a different operation and generally omits the latter.

Check Your Understanding

Answer each question using the definitions and formulas above. Short solutions follow the questions.

Two models have posterior probabilities $0.75$ and $0.25$, and predict a mean endpoint of 10 and 18 units, respectively. What is the model-averaged mean?
In one sentence, distinguish within-model predictive variance from between-model variance.
Two mechanisms assign probabilities $0.80$ and $0.20$ to an endpoint exceeding a threshold. Their posterior model probabilities are $0.40$ and $0.60$. What is the averaged event probability?
Why is the highest-posterior-probability model's prediction not generally the BMA prediction?
Name two conditions needed for a meaningful average across candidate mechanisms.

Solutions. (1) $0.75(10)+0.25(18)=7.5+4.5=12$ units. (2) Within-model variance describes predictive spread conditional on a particular mechanism; between-model variance describes the weighted spread of the mechanisms' predictive means around the averaged mean. (3) $0.40(0.80)+0.60(0.20)=0.32+0.12=0.44$, or 44%. (4) The leading model may have posterior probability below one, so using only its prediction discards the other models' weighted predictive distributions. (5) For example, the mechanisms must predict the same endpoint under the same intervention, and their model probabilities must be based on comparable observed evidence with stated prior model probabilities.