Posterior Model Probabilities with Unequal Prior Odds

Why Prior Odds Matter

When several PK/PD structures remain scientifically defensible, their relative support is not determined by the data alone. Bayesian model comparison combines each model's marginal likelihood, which measures how well the model predicts the observed data after averaging over its parameter prior, with the model's prior probability. If candidate models have unequal prior probabilities, the model with the larger marginal likelihood need not have the larger posterior probability.

For example, a turnover model may be considered more plausible before observing the current dataset than a more complex delayed-effect model, based on prior scientific knowledge. The data can still favor the delayed-effect model, but the evidence must be weighed against the starting prior odds. This tutorial focuses on calculating and interpreting that combination. It does not make posterior model probabilities into probabilities that a model is literally true: they are conditional on the specified candidate set, likelihoods and prior distributions.

Definition: Let $M_1,\ldots,M_K$ be a finite set of candidate models, with prior model probabilities $\pi_k=P(M_k)$, where $\pi_k\geq 0$ and $\sum_{k=1}^{K}\pi_k=1$. For observed data $y$, let $m_k(y)=p(y\mid M_k)$ be the marginal likelihood under model $M_k$, obtained by integrating its likelihood over its parameter prior. When $\sum_{k=1}^{K}\pi_k m_k(y)$ is positive and finite, the posterior probability of model $M_i$ is $$ P(M_i\mid y)=\frac{\pi_i m_i(y)}{\sum_{k=1}^{K}\pi_k m_k(y)}. $$

The model-specific priors on parameters are part of each $m_k(y)$; the $\pi_k$ are probabilities assigned to the models themselves. The candidate models must use the same observed data and a coherent probability specification. Marginal likelihoods must be defined, finite and comparable under those specifications. The prior model probabilities must be assigned before using the observed data being compared. A model with $\pi_i=0$ receives posterior probability zero whenever the denominator is positive, no matter how large its marginal likelihood is.

Two Models: Prior Odds Times a Bayes Factor

For two models $M_i$ and $M_j$, define the Bayes factor in favor of $M_i$ over $M_j$ as $BF_{ij}=m_i(y)/m_j(y)$, provided $m_j(y)>0$. The orientation matters: reversing the model order gives $BF_{ji}=1/BF_{ij}$, when both marginal likelihoods are positive. If both prior probabilities are positive, the posterior odds can be written as prior odds multiplied by this Bayes factor.

\frac{P(M_i\mid y)}{P(M_j\mid y)} = \frac{\pi_i}{\pi_j}BF_{ij}, \qquad \pi_i>0,\ \pi_j>0,\ m_i(y)>0,\ m_j(y)>0.

This identity follows by dividing the posterior-probability formula for $M_i$ by the formula for $M_j$: the common normalizing denominator cancels, leaving $\pi_i m_i(y)/[\pi_j m_j(y)]$. The requirement that both prior probabilities and both marginal likelihoods be positive is essential for the odds and Bayes factor in this expression to be defined.

In a two-model set with positive priors and positive marginal likelihoods, the posterior probability of $M_i$ exceeds one half exactly when its posterior odds exceed one. Equivalently, the evidence must exceed the reciprocal of the prior odds:

P(M_i\mid y)>\frac{1}{2} \quad\Longleftrightarrow\quad BF_{ij}>\frac{\pi_j}{\pi_i}.

This threshold is specific to a two-model comparison. With three or more candidates, $M_i$'s posterior probability exceeds one half only if its weighted marginal likelihood exceeds the combined weighted marginal likelihood of all the alternatives. A pairwise Bayes factor greater than one, or even greater than the pairwise prior-odds threshold, does not by itself establish that condition.

Worked Example: Evidence That Does Not Overcome Unequal Prior Odds

Suppose a hypothetical PK/PD analysis compares an immediate Emax model $M_1$ with a delayed-effect model $M_2$. Assign prior probabilities $\pi_1=0.20$ and $\pi_2=0.80$. For the same hypothetical data, suppose the marginal likelihoods are $m_1(y)=0.012$ and $m_2(y)=0.004$, in a common density scale.

Calculate the Bayes factor. With the stated orientation, evidence favoring $M_1$ over $M_2$ is $BF_{12}=m_1(y)/m_2(y)=0.012/0.004=3$.

Calculate the posterior probabilities. The unnormalized model weights are $\pi_1m_1(y)=0.20(0.012)=0.0024$ and $\pi_2m_2(y)=0.80(0.004)=0.0032$. Their sum is $0.0024+0.0032=0.0056$. Therefore, $P(M_1\mid y)=0.0024/0.0056=0.4286$, and $P(M_2\mid y)=0.0032/0.0056=0.5714$.

The data favor $M_1$ by a factor of 3 in marginal likelihood, but its prior odds were $0.20/0.80=0.25$. The posterior odds are $0.25(3)=0.75$, corresponding to posterior probability $0.75/(1+0.75)=0.4286$. Thus $M_1$ has the larger marginal likelihood but not the larger posterior probability. To exceed one half in this two-model set, $BF_{12}$ would need to exceed $\pi_2/\pi_1=0.80/0.20=4$.

More Than Two Candidate Models

For $K$ candidates, calculate every unnormalized weight $\pi_km_k(y)$, sum these weights, and divide each by that sum. The resulting probabilities are nonnegative and add to one when the normalizing sum is positive and finite. This direct formula is the safest starting point; pairwise odds are useful for comparisons, but do not replace normalization across the full candidate set.

Specify the candidate set and priors.
List each $M_k$, its prior probability $\pi_k$, and verify that the probabilities are nonnegative and sum to one.

Obtain comparable marginal likelihoods.
Use the same observed data and the appropriate parameter priors for each model. Check that the marginal likelihoods are finite and nonnegative.

Form and normalize the weights.
Calculate $w_k=\pi_km_k(y)$, then $P(M_k\mid y)=w_k/\sum_{\ell=1}^{K}w_\ell$, if the sum is positive and finite.

Interpret conditionally.
Report the posterior probabilities as support among the specified candidates, conditional on the data, models and prior choices used.

Worked Example: Normalizing Three PK/PD Model Weights

Consider a hypothetical comparison of three models for an exposure-response dataset: $M_1$, an immediate Emax model; $M_2$, a delayed-effect model; and $M_3$, a turnover model. Suppose the prior model probabilities are $0.20$, $0.50$ and $0.30$, respectively, and the marginal likelihoods are $0.010$, $0.004$ and $0.002$, respectively.

The unnormalized weights are $w_1=0.20(0.010)=0.0020$, $w_2=0.50(0.004)=0.0020$, and $w_3=0.30(0.002)=0.0006$. The normalizing sum is $0.0020+0.0020+0.0006=0.0046$.

Thus $P(M_1\mid y)=0.0020/0.0046=0.4348$, $P(M_2\mid y)=0.0020/0.0046=0.4348$, and $P(M_3\mid y)=0.0006/0.0046=0.1304$. The probabilities sum to $0.4348+0.4348+0.1304=1.0000$, up to rounding. Although $M_2$ had the largest prior probability, its posterior probability ties that of $M_1$: its smaller marginal likelihood offsets its larger prior. Neither model has posterior probability above one half because the alternatives collectively retain substantial weighted support.

Boundary Cases and Numerical Checks

Zero values require direct use of the posterior-probability formula, not odds identities that divide by zero. A zero prior probability cannot be overcome by the data: its unnormalized weight is zero. A zero marginal likelihood gives zero weight to that model if its prior is finite. If a model's marginal likelihood is positive but its prior is zero, it still receives posterior probability zero. These cases are different from a positive prior and positive evidence, for which the usual odds calculation applies.

Worked Example: Zero Prior Probability and Zero Evidence

Suppose a hypothetical three-model set has priors $\pi_1=0$, $\pi_2=0.60$, and $\pi_3=0.40$. For the observed data, let $m_1(y)=0.020$, $m_2(y)=0$, and $m_3(y)=0.005$. These values are illustrative; in an application, a computed zero may also indicate a numerical underflow that needs investigation.

The weights are $w_1=0(0.020)=0$, $w_2=0.60(0)=0$, and $w_3=0.40(0.005)=0.002$. The sum is $0.002$, so the posterior probabilities are $0/0.002=0$, $0/0.002=0$, and $0.002/0.002=1$. Model $M_1$ cannot receive posterior support because its prior is zero, despite its positive marginal likelihood. Model $M_2$ receives zero because its marginal likelihood is zero. The remaining model $M_3$ receives posterior probability one within this candidate set.

If every model with positive prior probability has zero marginal likelihood, then $\sum_k\pi_km_k(y)=0$ and the posterior model probabilities are undefined under this specification. This can mean that the observed data are outside the support of all models receiving prior mass, or that likelihood calculations failed. It is not a case in which one should divide by zero or silently assign equal probabilities.

In computation, marginal likelihoods can be extremely small. Dividing tiny floating-point numbers may cause underflow. A standard remedy is to work with log weights, $a_k=\log(\pi_k)+\log(m_k(y))$, for positive $\pi_k$ and $m_k(y)$, and then normalize after subtracting the largest $a_k$. Zero weights are handled separately, with log weight negative infinity. The final probabilities should still be checked to ensure they are nonnegative and sum to one within rounding tolerance.

Interpretation, Diagnostics, and Alternatives

Posterior model probabilities answer a conditional question: given the specified candidate models, their parameter priors and model prior probabilities, and the observed data, how is posterior probability distributed across these models? They do not measure the probability that a model is true without qualification. If scientifically relevant structures are omitted, the probabilities still sum to one over the included candidates; that normalization does not establish that the set is adequate.

A useful diagnostic is to display prior probabilities, marginal likelihoods or Bayes factors, and posterior probabilities together. This makes it possible to see whether a posterior ranking is driven mainly by prior odds or by data evidence. In line with the earlier material on prior sensitivity in Bayesian model comparison, repeat calculations under defensible alternative model priors when conclusions may depend strongly on their allocation. Do not select priors after examining the same data and then describe the resulting probabilities as if those priors were specified independently.

Another check is whether all models use the same data definition and likelihood scale. Marginal likelihoods must be calculated under each model's parameter prior; substituting maximized likelihoods does not produce the same quantities. Very small or unexpectedly large marginal likelihoods, probabilities that fail to sum to one, or results that change materially under numerical tolerances call for checking integration, parameter-prior specification, and data handling before interpretation.

Approach	What it summarizes	How it differs from posterior model probabilities
Bayesian posterior model probabilities	Prior model weights updated by marginal likelihoods	Depend on model priors and parameter priors; are normalized probabilities conditional on the candidate set.
AICc or BIC	Penalized summaries based on likelihood and model complexity	Are not posterior probabilities and do not directly combine with prior model probabilities using the posterior formula.
Subject-level cross-validation	Predictive performance on held-out subjects or data partitions	Evaluates predictive performance under a specified validation design; it does not produce the same prior-updated model probabilities.

These approaches can inform different questions and may provide complementary evidence. Their outputs should not be relabeled as posterior probabilities or combined as though they were interchangeable. Posterior model probabilities also have limitations: they can be sensitive to parameter priors through the marginal likelihood, to model priors directly, and to which candidates are included. The numbers quantify relative support under those choices, not protection against misspecification or poor extrapolation.

Common Mistake

Wrong reasoning: “The Bayes factor for $M_i$ over $M_j$ exceeds the reciprocal prior-odds threshold, so $M_i$ must have posterior probability above one half,” without checking whether the comparison is truly between only two models or whether both priors and marginal likelihoods are positive.

Correction: The threshold $BF_{ij}>\pi_j/\pi_i$ gives the greater-than-one-half condition only for two models with $\pi_i,\pi_j>0$ and $m_i(y),m_j(y)>0$. For a larger set, compare $\pi_i m_i(y)$ with the sum of all other weighted marginal likelihoods. If a prior or marginal likelihood is zero, calculate the normalized weights directly; do not use an undefined odds ratio or Bayes factor.

Key takeaway: Posterior model probabilities are normalized products of prior model probabilities and marginal likelihoods. Unequal priors affect the evidence needed to change a model's standing, while zero-prior and zero-evidence cases must be treated directly through the weight formula.

Check Your Understanding

Use the stated Bayes-factor orientation and show how prior odds and evidence combine. Solutions are included below.

In a two-model comparison, $\pi_1=0.25$, $\pi_2=0.75$, and $BF_{12}=6$. Calculate both posterior probabilities.
For three models, the prior probabilities are $0.50$, $0.30$, and $0.20$; the marginal likelihoods are $0.004$, $0.010$, and $0.005$. Calculate the posterior probability of each model.
Why is the condition $BF_{ij}>\pi_j/\pi_i$ not valid as written when $\pi_i=0$?
For three candidates, $w_i=0.003$ and the combined weight of the other candidates is $0.004$. Is $P(M_i\mid y)$ greater than one half? Explain.

Solutions. (1) The prior odds are $0.25/0.75=1/3$; posterior odds are $(1/3)(6)=2$. Hence $P(M_1\mid y)=2/(1+2)=0.6667$ and $P(M_2\mid y)=0.3333$. (2) The weights are $0.50(0.004)=0.0020$, $0.30(0.010)=0.0030$, and $0.20(0.005)=0.0010$, summing to $0.0060$. The posterior probabilities are $0.3333$, $0.5000$, and $0.1667$. (3) The ratio $\pi_j/\pi_i$ is undefined when $\pi_i=0$, and posterior odds involving $M_i$ are also undefined. Use the normalized weight formula; with a positive denominator, $M_i$'s posterior probability is zero. (4) No. Its posterior probability is $0.003/(0.003+0.004)=3/7=0.4286$, which is below one half because its weight is less than the combined weight of the alternatives.