Simulating Trial Replicates From a Prior

What a Simulated Trial Replicate Represents

A prior predictive check asks what trial data could look like before the trial is observed, given both the design and the assumptions in the prior and sampling model. A simulated trial replicate is one complete hypothetical dataset generated under those assumptions. It is not a new estimate from observed trial data, and it is not a posterior predictive dataset: no trial outcomes have yet been used to update the prior.

The key is to reproduce the design as well as the outcome model. If the protocol plans 120 participants in each of two arms and records one binary primary endpoint for every participant, a complete replicate has those 240 participant records, the planned arm labels, and the binary endpoint values. Generating just two event counts may be enough for some questions, but it is not a complete dataset if the design also records participant-level or additional endpoint information.

Definition: A prior predictive trial replicate is a dataset generated by first drawing the trial’s unknown parameters jointly from their specified prior distribution and then drawing all planned outcomes from the sampling model conditional on those parameters and the trial design. Within one replicate, parameters shared by participants are drawn once and used for all relevant outcomes.

The prior draw is trial-level: it represents one possible underlying state of the world for the hypothetical trial. Outcome draws are then made at the participant or observation level as specified by the model. Drawing a new treatment effect independently for every participant would usually describe a different model, not a replicate from the planned trial model.

A Generative Recipe

Let $d$ denote the fixed design, including planned arm sizes and the endpoint structure. Let $\theta$ collect all unknown model parameters, and let $\tilde y$ denote the complete hypothetical dataset. The simulation follows the joint prior $p(\theta)$ and sampling model $p(\tilde y\mid\theta,d)$. The resulting prior predictive distribution is their mixture:

p(\tilde y\mid d)=\int p(\tilde y\mid\theta,d)\,p(\theta)\,d\theta.

In practice, one usually cannot enumerate every possible dataset or evaluate this integral for every question of interest. Simulation approximates it by repeating the two-stage draw. In each repetition, draw a fresh $\theta$ from the joint prior, then generate the complete dataset conditional on that draw. The design $d$ is held fixed unless the simulation question explicitly includes randomizing some design feature.

Write down the design.
Specify planned arm sizes, allocation, participants or units, endpoint variables, and any repeated-measure or grouping structure that the model includes.

Draw the joint parameters.
Generate one draw from the prior for each parameter needed in the trial. Preserve any dependence specified among parameters by drawing from their joint prior, rather than replacing it with independent draws.

Generate outcomes under the sampling model.
Use the drawn parameters and the planned design to generate every required outcome. Respect the model’s conditional dependence assumptions.

Store the complete replicate.
Retain participant identifiers, assigned arms, endpoint values, and any relevant design variables. Repeat the full procedure independently to obtain many replicates.

If outcomes are conditionally independent given $\theta$ and $d$, the likelihood for a complete replicate factors into participant-level contributions. That factorization is a model assumption, not an automatic property of simulation. If the sampling model includes shared clustering, repeated observations, or dependence between endpoints, the simulation must preserve that structure instead of generating each value as if it were unrelated.

Justification: If $\theta^{(r)}$ is drawn from $p(\theta)$ and $\tilde y^{(r)}$ is then drawn from $p(\tilde y\mid\theta^{(r)},d)$, the probability of a dataset region $A$, averaged over repetitions, estimates $P(\tilde y\in A\mid d)$. For $R$ independent replicates, the estimate is $$ \widehat{P}(\tilde y\in A\mid d)=\frac{1}{R}\sum_{r=1}^{R} I\{\tilde y^{(r)}\in A\}, $$ where $I\{\cdot\}$ equals 1 when the event is true and 0 otherwise. This follows because each simulated dataset has the prior predictive distribution obtained by integrating the conditional sampling distribution over the prior.

Worked Replicates

Worked Example: A Binary Endpoint in Two Fixed Arms

Consider a hypothetical trial with 3 participants assigned to control and 3 to treatment. The endpoint is an indicator of whether a participant experiences an event by week 8. Assume $Y_{ai}\mid p_a\sim\operatorname{Bernoulli}(p_a)$, independently across participants conditional on the arm probabilities, where $a$ is the arm and $i$ indexes its participants. Suppose the prior distributions are $p_C\sim\operatorname{Beta}(2,8)$ and $p_T\sim\operatorname{Beta}(2,6)$, independently. For illustration, take one random prior draw to be $p_C=0.20$ and $p_T=0.30$. These values are one hypothetical draw, not fixed prior means.

To generate outcomes, draw independent uniform values on $[0,1]$ and set $Y_{ai}=1$ if the value is less than $p_a$, and $Y_{ai}=0$ otherwise. Suppose the control-arm values are $0.12, 0.34, 0.81$, and the treatment-arm values are $0.09, 0.27, 0.64$. For control, $0.12<0.20$ gives an event and the other two values exceed $0.20$, so its outcomes are $(1,0,0)$. For treatment, $0.09<0.30$ and $0.27<0.30$, while $0.64>0.30$, giving $(1,1,0)$.

The complete replicate therefore has 6 participant records, with the planned 3:3 allocation and binary outcomes in both arms. The event counts are 1 of 3 in control and 2 of 3 in treatment. Their difference in observed proportions, treatment minus control, is $2/3-1/3=1/3$, or 33.3 percentage points. That is a summary of this particular hypothetical replicate, not a treatment-effect estimate based on real trial data.

Worked Example: A Continuous Endpoint With a Shared Scale Parameter

Suppose a hypothetical trial plans 3 participants per arm and records change in systolic blood pressure at week 12, in mmHg. Use the model $Y_{ai}\mid\mu_a,\sigma\sim N(\mu_a,\sigma^2)$, conditionally independently across participants. Let the priors be $\mu_C\sim N(100,15^2)$, $\delta\sim N(-5,8^2)$, and $\sigma=20$ mmHg, with $\mu_T=\mu_C+\delta$. For one hypothetical prior draw, take $\mu_C=108$ mmHg and $\delta=-12$ mmHg, so $\mu_T=108+(-12)=96$ mmHg. The residual standard deviation is 20 mmHg.

For each participant, draw a standard normal value $Z$ and calculate $Y_{ai}=\mu_a+\sigma Z$. Suppose the control draws are $Z=(-0.5,0.2,1.1)$. The outcomes are $108+20(-0.5)=98$, $108+20(0.2)=112$, and $108+20(1.1)=130$ mmHg. Suppose the treatment draws are $Z=(-1.0,0.4,0.75)$. The outcomes are $96+20(-1.0)=76$, $96+20(0.4)=104$, and $96+20(0.75)=111$ mmHg.

This replicate contains all 6 planned continuous outcomes and the fixed arm sizes. Its arm means are $340/3=113.3$ mmHg in control and $291/3=97.0$ mmHg in treatment, a replicate-level difference of $-16.3$ mmHg for treatment minus control. The difference need not equal the drawn prior treatment contrast of $-12$ mmHg: participant outcomes also vary around their arm means.

Worked Example: Retaining Two Endpoints Per Participant

Suppose a hypothetical trial plans 2 participants in each arm and records both a continuous symptom score at week 4 and a binary indicator of improvement. To illustrate a model with conditionally independent endpoints, let $X_{ai}\mid\mu_a\sim N(\mu_a,2^2)$ points and $B_{ai}\mid q_a\sim\operatorname{Bernoulli}(q_a)$, independent given the parameters. Draw the parameters once for this replicate: $\mu_C=10$, $\mu_T=8$ points, $q_C=0.30$, and $q_T=0.50$. These values are assumed hypothetical prior draws.

Take the standard normal draws for the continuous outcomes to be $0.5,-1.0$ in control and $0.0,1.0$ in treatment. The resulting scores are $10+2(0.5)=11$, $10+2(-1)=8$, $8+2(0)=8$, and $8+2(1)=10$ points, respectively. For the binary endpoint, take uniform draws $0.20,0.70$ in control and $0.35,0.90$ in treatment. Comparing each value with its arm probability gives control improvement indicators $(1,0)$ and treatment indicators $(1,0)$.

A complete replicate has four participant rows, and each row contains both the continuous score and the binary outcome for the same participant. The design and endpoint structure are intact. Conditional independence between endpoints was an explicit assumption in this example; if a trial model specifies dependence, the simulation must use that joint outcome model instead.

Monte Carlo Summaries and Practical Checks

After generating many complete replicates, calculate the same summaries that are relevant to the planned trial: for example, an arm mean, an event count, or a treatment-minus-control contrast. The distribution of those summaries across replicates describes their implications under the prior predictive model. A simulated interval or quantile describes variation across hypothetical datasets under that model; it is not a confidence interval from observed data or a posterior credible interval for a parameter.

Several implementation checks help detect a simulation that does not match its specification:

Check the design in every replicate. Verify the number of records per arm, arm labels, and the presence of every planned endpoint. A dropped endpoint or accidental change in arm size changes the question being simulated.
Check the parameter-draw level. Confirm that a parameter meant to be shared by an arm or trial is not being redrawn separately for each participant. Conversely, participant-level random effects should be generated at the level specified by the model.
Check the joint prior. If the prior specifies dependent parameters, use a joint draw. Sampling each marginal independently removes that dependence and may produce a different predictive distribution.
Check the outcome-generating step. Review the distribution, parameterization, units, and conditioning used in the code. For a normal distribution, for example, software may request a standard deviation or a variance; entering 20 where a variance is expected would not implement the model in the example.
Check summaries against individual records. Recompute a few arm counts or means from the stored dataset and compare them with the values reported by the simulation code.

A fixed random-number seed can make a simulation run reproducible, but it does not establish that the code or model is correct. Reproducibility means that the same specified computation can be repeated; validation also requires checking that the computation implements the intended design, prior, and sampling model.

How This Approach Differs From Alternatives

Approach	What it generates or calculates	Useful distinction
Complete prior predictive simulation	Parameter draws followed by full hypothetical datasets	Retains participant records, planned allocation, and endpoint structure.
Analytical prior predictive calculation	Direct probabilities or distributions derived by integrating the model	Can be efficient when a tractable result is available, but may not readily provide complex datasets or summaries.
Summary-only simulation	Counts, means, or other aggregates without storing every participant outcome	May answer a targeted summary question, but cannot support checks that require the complete endpoint structure.
Posterior predictive simulation	Datasets generated using a parameter distribution updated with observed outcomes	Addresses predictions after observing data; prior predictive simulation uses the prior before those outcomes are observed.

The complete-data approach is valuable when the intended checks involve more than a single summary, including whether simulated datasets have the planned shape and whether summaries can be recomputed consistently from participant records. It also makes computational costs and model assumptions visible. Its limitation is that simulation only explores what the specified model can generate: a large number of replicates cannot compensate for an incorrectly stated prior, sampling distribution, or design.

Common Mistakes

Redrawing shared parameters for each participant. This changes a trial-level uncertainty into participant-to-participant parameter variation. Draw each shared parameter once per replicate, then generate all outcomes that depend on it.
Drawing a parameter once and reusing it forever. This produces repeated outcomes conditional on one parameter draw, rather than a sample from the prior predictive distribution. Draw a fresh parameter set for each independent replicate.
Generating only the endpoint summary. An event count alone does not retain participant identifiers, arm assignment, or additional endpoints. Store the complete dataset when complete trial replicates are the target.
Calling the simulated treatment contrast the true effect. A contrast calculated from one replicate includes outcome variation and depends on one prior draw. It is a feature of that hypothetical dataset, not a known clinical effect.

Key takeaway: One prior draw defines a hypothetical parameter state for one trial replicate. Conditional on that state, generate all outcomes under the sampling model while preserving the planned design and endpoint structure; then draw a new parameter state for the next replicate.

Check Your Understanding

Answer each question, then compare your reasoning with the short solutions.

A trial has 40 participants per arm. A shared arm-specific event probability is part of the model. Should it be drawn once per participant or once per simulated trial replicate?
For $R=500$ simulated replicates, 125 have an event count above 30. What Monte Carlo estimate does this give for the prior predictive probability that the count exceeds 30?
A protocol records a continuous and a binary endpoint on every participant. What must a complete replicate retain?
Why can a treatment-minus-control contrast in one simulated dataset differ from the prior-drawn treatment contrast?

Solutions. (1) Draw the shared probability once per replicate and use it for all relevant participants; draw a new probability for the next replicate. (2) The estimate is $125/500=0.25$, or 25%. (3) It should retain the planned participant records and arm labels, with both endpoint values for each participant. (4) The dataset contrast includes outcome-level sampling variation around the parameters drawn for that replicate, so it need not equal the parameter contrast.