Comparing Evidence Across Causal Models

Why Comparing Causal Models Requires More Than a Strong Association

Fine-mapping compares explanations of the observed genetic data, not just variants ranked by association strength. A causal configuration specifies which candidate variants have nonzero effects under a model. The data may fit one configuration better than another, but the posterior probability of that configuration also depends on how probable it was before observing the data. This distinction matters especially when asking how many causal signals are supported: a model with exactly one causal variant is a collection of possible configurations, not a single competitor.

The key quantity for comparing models is the marginal likelihood: the likelihood of the observed data averaged over the parameter values allowed by a model and weighted by their prior distribution. A Bayes factor is a ratio of marginal likelihoods. A posterior model probability combines that data-based comparison with the prior model probabilities.

Configurations, Marginal Likelihoods, and Posterior Probabilities

Suppose the candidate configurations are mutually exclusive and exhaustive, with prior probabilities \(\pi_\gamma=P(\gamma)\). Bayes’ rule gives \(P(\gamma\mid D)=m_\gamma(D)\pi_\gamma/\sum_{\gamma'}m_{\gamma'}(D)\pi_{\gamma'}\). Dividing every marginal likelihood by the reference marginal likelihood yields an equivalent calculation using Bayes factors:

For two configurations \(\gamma_a\) and \(\gamma_b\), this implies \(P(\gamma_a\mid D)/P(\gamma_b\mid D) =[P(\gamma_a)/P(\gamma_b)]BF_{\gamma_a,\gamma_b}\). Thus a Bayes factor of 10 favors \(\gamma_a\) over \(\gamma_b\) by a factor of 10 in the data evidence; it does not, by itself, say that \(\gamma_a\) has posterior probability 0.90. The prior odds and the other models in the comparison also matter.

From Configuration Evidence to Evidence About Signal Count

Write \(K=|\gamma|\) for the number of causal variants in configuration \(\gamma\). Let \(\rho_k=P(K=k)\) be the prior probability of \(k\) signals, and let \(q(\gamma\mid k)\) be the conditional prior probability of configuration \(\gamma\) among the configurations with \(k\) variants. For each \(k\), the conditional probabilities must sum to one over configurations of size \(k\). The marginal evidence for count \(k\), relative to the reference model, is the prior-weighted average \(BF_k=\sum_{|\gamma|=k}q(\gamma\mid k)BF_{\gamma,\gamma_0}\). Consequently, \(P(K=k\mid D)\) is proportional to \(\rho_k BF_k\), not to the Bayes factor of the single best configuration of size \(k\).

This aggregation keeps two questions separate. First, how well does each configuration explain the data after averaging over its parameters? Second, how much prior probability was assigned to that configuration or signal count? Within a count, posterior configuration probabilities are proportional to \(q(\gamma\mid k)BF_{\gamma,\gamma_0}\). Across counts, the prior \(\rho_k\) also enters.

Why the Marginal Evidence Averages Across Configurations

The averaging in \(BF_k\) follows directly from the law of total probability. Conditional on \(K=k\), the probability of the data under that count model is the sum of the probabilities under its possible configurations, weighted by their conditional prior probabilities:

Dividing both sides by the reference marginal likelihood \(m_{\gamma_0}(D)\) gives the stated expression for \(BF_k\). The weights \(q(\gamma\mid k)\) must be included: simply summing configuration Bayes factors would make the count’s evidence depend mechanically on how many configurations happen to be listed. The average instead represents the model in which a configuration is drawn from the specified conditional prior and then generates the data.

Marginal likelihoods also average over effect parameters, rather than inserting only their best-fitting values. A model can fit well at a narrow range of effect values but assign little prior mass to that range; its marginal evidence reflects both fit and the parameter prior. The prior on effect sizes must be proper for the ordinary marginal likelihood and Bayes factor above to be defined. The earlier material on sensitivity to effect-size priors is relevant because changing that prior can change the Bayes factor, even when the likelihood and observed data are unchanged.

Configuration Priors Matter Within a Signal Count

Even when the prior probability on the number of signals is fixed, the allocation of prior probability among configurations of that size affects the count-level evidence. For example, if one variant has a much larger Bayes factor than another, assigning it more conditional prior probability increases the prior-weighted average \(BF_k\). This is not an arithmetic nuisance: it is the consequence of defining a different prior model.

Assumptions, Diagnostics, and Alternatives

The comparison is interpretable only relative to a clearly specified model set. The candidate configurations should be mutually exclusive and collectively exhaustive for the comparison being reported, and their prior probabilities must be nonnegative and sum to one. Conditional configuration priors must also sum to one within each signal count. Each model’s likelihood must describe the same observed data, including the same treatment of sample, phenotype, and LD information; otherwise marginal likelihood ratios do not isolate model differences. The likelihood and priors should reflect the intended statistical assumptions, and parameter priors used for marginal likelihoods must be proper.

Useful checks include verifying that prior probabilities and conditional weights normalize, recalculating posterior weights from \(\pi_\gamma BF_{\gamma,\gamma_0}\), and confirming that posterior probabilities sum to one. Report whether evidence is for a particular configuration or for an aggregated signal count, and state the priors used for both. If a conclusion about count changes substantially under plausible count or configuration priors, that dependence is part of the result and should be made visible. Sensitivity to LD assumptions or effect-size priors can also alter configuration evidence; a posterior probability should not be presented as though it were prior-free.

A p-value addresses how unusual a test statistic would be under a specified null model; it is not a probability assigned to a causal configuration. Information criteria such as AIC or BIC provide alternative model-ranking or approximation approaches, but they are not the same calculation as posterior model probabilities with explicit prior model weights. Neither a small p-value nor the largest Bayes factor automatically gives the posterior probability of a model. Posterior probabilities require normalization across the specified model set.

A common mistake is to say, “The two-signal model has the largest Bayes factor, so the posterior probability of two signals is largest.” This reasoning can fail twice: the Bayes factor may describe only the best configuration rather than the prior-weighted evidence across all two-signal configurations, and the count’s prior probability has not been included. The correction is to calculate \(BF_k\) by averaging over configurations using \(q(\gamma\mid k)\), multiply by \(\rho_k\), and normalize across the compared counts.