Prior Predictive Checks for Knowledge-Informed Models

Why Check the Prior Before Seeing the Data?

A biological-knowledge prior can encode pathway membership, interaction confidence, or directional relationships between molecular layers. But a prior that sounds reasonable in terms of its individual parameters may imply implausible data: pathway contrasts that are almost always enormous, a nearly complete interaction network, or far too many cross-layer links. A prior predictive check makes those consequences visible by simulating from the model before conditioning on the observed omics measurements.

The object being checked is not whether a prior is biologically true. It is whether the model, with its chosen prior and data-generating assumptions, can produce outcomes and structures that are plausible for the scientific setting. For knowledge-informed models, this includes checking implications at the level biological knowledge is meant to inform: pathway effects, network density, and cross-layer connectivity.

Definition: Let $\theta$ denote all unknown parameters, $Y$ the data, $p(\theta)$ the prior distribution, and $p(Y\mid\theta)$ the likelihood. The prior predictive distribution is $$ p(Y)=\int p(Y\mid\theta)p(\theta)\,d\theta. $$ It describes the distribution of possible data under the model before the observed data are used to update beliefs about $\theta$.

A prior predictive simulation draws $\theta^{(m)}$ from $p(\theta)$, then draws $Y^{(m)}$ from $p(Y\mid\theta^{(m)})$, for each simulation $m$. Any model quantities needed for checks—such as pathway contrasts or edge indicators—are recorded along the way. The observed data can later be compared with these draws, but that comparison is not needed to check the prior’s implications.

Model Assumptions and a Simulation Plan

Consider a model with gene-level effects organized by pathway, molecular interaction indicators, and links between two omics layers. The exact distributions depend on the application. The checks below assume that the specified prior distributions and likelihood describe the intended model, that the simulation draws follow those distributions, and that the quantities being summarized have been defined consistently with the scientific question. For instance, a network density requires a defined set of candidate edges; an absent edge outside that set is not the same as a candidate edge assigned probability zero.

Before simulating, specify interpretable quantities and plausible ranges. A pathway effect might be summarized by its average standardized contrast; a network by its edge count and density; and cross-layer structure by the number of links or the distribution of node degrees. Standardization can help compare features measured in different units, but it does not make biological scales interchangeable. If effects are not standardized, retain their original units when interpreting results.

Draw prior parameters.
Sample pathway effects, edge probabilities or edge indicators, cross-layer coefficients, and other unknown parameters from their specified joint prior. Preserve dependencies between parameters; do not replace a joint draw with unrelated marginal draws.

Generate model quantities and data.
Use the model’s conditional distributions to generate gene effects, network edges, cross-layer links, and, where defined, omics measurements. Follow the stated likelihood, including its noise and dependence assumptions.

Summarize implications.
For each draw, calculate prespecified quantities such as pathway-effect magnitude, network density, or cross-layer link count. Examine ranges and tail probabilities, not only averages.

Compare with scientific constraints.
Ask whether the simulated quantities are credible for the intended tissue, context, and measurement scale. Investigate surprising results in the model specification and implementation before interpreting any data analysis.

Worked Checks for Pathways and Networks

Worked Example: Pathway Effects on a Standardized Scale

Suppose a hypothetical pathway contains four genes. Its shared effect is $b\sim N(0,0.8^2)$, and gene $j$'s effect is $\beta_j=b+u_j$, where the $u_j$ are independent $N(0,0.3^2)$ deviations, also independent of $b$. The pathway average is $\bar{\beta}_{P}=\frac{1}{4}\sum_{j=1}^4\beta_j$. For a prior predictive measurement, suppose the average observed contrast is $D_P\mid\bar{\beta}_{P}\sim N(\bar{\beta}_{P},0.25^2)$. All effects and contrasts here are hypothetical and standardized.

The pathway average is $b+\frac{1}{4}\sum_j u_j$. Independence gives $$ \operatorname{Var}(\bar{\beta}_{P})=0.8^2+\frac{0.3^2}{4}=0.64+0.0225=0.6625. $$ Adding the measurement variance gives $\operatorname{Var}(D_P)=0.6625+0.25^2=0.725$, so $D_P\sim N(0,0.725)$ and its standard deviation is approximately $0.8515$.

A useful check is the prior probability that the observed pathway-average contrast exceeds one standardized unit in magnitude: $$ P(|D_P|>1)=2\left[1-\Phi\left(\frac{1}{\sqrt{0.725}}\right)\right] =2[1-\Phi(1.174)]\approx 0.240. $$ Thus the model implies about a 24.0% probability of a contrast beyond one standardized unit, before seeing data. Whether this is plausible depends on the scientific context; the calculation is a diagnostic, not a universal threshold.

One possible prior draw is $b=0.40$, with deviations $0.10,-0.20,0.05,0.05$. The gene effects are $0.50,0.20,0.45,0.45$, with average $0.40$. If the measurement-noise draw is $0.15$, then $D_P=0.40+0.15=0.55$. This single draw illustrates the simulation mechanics; the probability calculation summarizes the full prior predictive distribution.

Worked Example: Implied Network Density

Suppose a hypothetical undirected network has ten candidate edges: four supported by a knowledge source and six without that annotation. Conditional on the specified probabilities, assume the edge indicators are independent Bernoulli variables, with inclusion probability $0.80$ for each annotated edge and $0.05$ for each unannotated edge. This independence assumption is made for this example; a model with dependent edges would need simulations that retain that dependence.

Let $E$ be the number of included edges and let density be $D=E/10$. Linearity of expectation gives $$ E(E)=4(0.80)+6(0.05)=3.50,\qquad E(D)=\frac{3.50}{10}=0.350. $$ The edge-count variance is the sum of the Bernoulli variances: $$ \operatorname{Var}(E)=4(0.80)(0.20)+6(0.05)(0.95)=0.925. $$ Therefore the standard deviation of density is $\sqrt{0.925}/10\approx0.0962$. The prior does not imply that every simulated network has density 0.350; it implies a distribution of densities around that mean.

For one illustrative draw, suppose three of the four annotated edges and one of the six unannotated edges are included. Then $E=3+1=4$, and the simulated density is $D=4/10=0.400$. A prior predictive check would repeat this draw many times and inspect the density distribution, edge counts, and possibly node degrees. If even typical simulations are denser than any network considered plausible in the application, the model’s implications need examination.

Cross-Layer Links and Simulation Precision

Worked Example: Prior Cross-Layer Link Counts

Suppose a hypothetical model considers eight candidate links between two molecular layers. For each link $r$, let $L_r$ be its inclusion indicator and suppose $L_r\mid z_r\sim\operatorname{Bernoulli}(p_r)$, where $\operatorname{logit}(p_r)=-2+1.5z_r$. Here $z_r=1$ for three high-confidence candidate links and $z_r=0$ for five other candidates. Assume the indicators are conditionally independent given these probabilities.

The high-confidence link probability is $\operatorname{logit}^{-1}(-0.5)=1/(1+e^{0.5})\approx0.3775$. The other probability is $\operatorname{logit}^{-1}(-2)=1/(1+e^2)\approx0.1192$. If $L=\sum_{r=1}^8L_r$, then $$ E(L)=3(0.3775)+5(0.1192)=1.729, \qquad E(L/8)=1.729/8=0.216. $$ Thus the prior implies an average of about 1.73 links, or a density of about 0.216, over this defined candidate set.

For example, one simulated draw might include one of the three high-confidence links and none of the other five. It has $L=1$ and density $1/8=0.125$. Such a draw is not a contradiction of the expected density: individual simulations vary. Repeated simulations reveal how often the model produces no links, many links, or highly uneven link patterns.

Monte Carlo summaries have their own uncertainty. If $M$ independent prior predictive draws yield $K$ draws satisfying a check such as $|D_P|>1$, the estimated probability is $\hat q=K/M$. Its approximate Monte Carlo standard error is $\sqrt{\hat q(1-\hat q)/M}$. For example, if $\hat q=0.240$ from $M=5{,}000$ draws, this standard error is $\sqrt{0.240(0.760)/5000}\approx0.00604$. Increasing $M$ reduces simulation error; it does not resolve uncertainty about whether the model itself is appropriate.

Justification and Practical Diagnostics

Prior predictive simulation works because averages and variances under the predictive distribution combine uncertainty in the parameters with variation in the data given those parameters. For any scalar quantity $Y$ with finite variance, the law of total expectation and variance gives $$ E(Y)=E_{\theta}\{E(Y\mid\theta)\}, \qquad \operatorname{Var}(Y)=E_{\theta}\{\operatorname{Var}(Y\mid\theta)\}+ \operatorname{Var}_{\theta}\{E(Y\mid\theta)\}. $$ The first term in the variance expression is average sampling variation; the second is variation in the conditional mean induced by the prior. Simulation reproduces both components by drawing $\theta$ first and $Y$ second. This also explains why simulating data at a single “typical” parameter value is not a prior predictive check: it omits variation across the prior.

Useful diagnostics include plotting simulated pathway summaries, tabulating edge counts and densities, examining degree distributions, and checking cross-layer link counts. Compare lower and upper tails as well as central values. In a multi-omics model, inspect each layer and relevant combinations: reasonable marginal distributions can still conceal implausible dependence or an implausible concentration of links on a few nodes.

Check the implementation. Confirm that sampled indicators use the intended probabilities, that undirected edges are not counted twice, and that pathway membership and candidate-link sets have the intended dimensions.
Check units and transformations. A prior on standardized effects implies different numerical contrasts from a prior on raw expression units. Verify that the simulated likelihood uses the same scale as the stated prior.
Check dependence. If pathway effects share a parent parameter, or edge indicators are dependent, independent marginal draws will give the wrong joint prior predictive distribution.
Check what “density” means. State the denominator—the number of candidate edges or links—because changing the candidate set changes the density even when the included-edge count is unchanged.

Interpretation, Alternatives, and Limitations

Prior predictive checks differ from posterior predictive checks. A prior predictive check integrates over the prior before observing the current data; a posterior predictive check instead simulates using the posterior distribution after conditioning on those data. The first asks whether the prior-plus-likelihood makes plausible predictions at all. The second assesses aspects of fit after the data have updated the parameter distribution. Neither establishes that a model is biologically correct.

They also complement, rather than replace, direct elicitation of prior parameters, checks of database quality, and model-based alternatives such as imposing hard pathway constraints. Directly inspecting parameter values may miss their combined consequences, while prior predictive simulation shows those consequences on data- or science-facing scales. It can reveal an implausible model even if each parameter’s marginal prior appears defensible.

A prior predictive check has limits. Biological plausibility is context-dependent and cannot be decided by a generic numerical cutoff. A model can generate plausible pathway effects and network density while missing important biology, such as unrepresented pathways or dependence across layers. Conversely, an unusual simulated feature may reflect a genuine but rare possibility rather than a defect. Record the scientific rationale for any plausibility judgment. This exercise evaluates implications of the specified prior; it does not vary prior choices to assess their stability.

Common pitfall: “The average simulated network density is plausible, so the network prior is plausible.” A reasonable mean does not rule out an excessive probability of nearly empty or nearly complete networks, nor implausible degree concentration. Inspect the distribution and relevant structural summaries, not just the mean.

Check Your Understanding

Answer the questions, then compare with the solutions below.

In a prior predictive simulation, why must parameters be drawn from the prior before data are generated?
A model has six candidate edges, each included independently with probability $0.10$. What is the expected edge count and expected density?
Why can a plausible mean cross-layer link count fail to establish that the link prior has plausible implications?
What is the distinction between a prior predictive check and a posterior predictive check?

Solutions. (1) Drawing parameters first and then data from the likelihood includes both prior parameter uncertainty and conditional data variation. Fixing parameters at one value omits the former. (2) The expected count is $6(0.10)=0.600$ edges; with six candidate edges, the expected density is $0.600/6=0.100$. (3) The same mean can arise from very different distributions. A prior might put substantial probability on both nearly empty and unusually dense networks, or concentrate links on implausibly few nodes; inspect the full distribution and relevant structure. (4) A prior predictive check simulates before conditioning on the observed data, using the prior and likelihood. A posterior predictive check simulates after updating parameter uncertainty with the observed data, using the posterior and likelihood.