Why Check Predictive Support?
A prior predictive distribution describes data that could arise before the trial data are observed, given the prior and the sampling model. Simulating trial replicates from that distribution can reveal whether the model produces outcomes that do not make sense for the endpoint or the design. This check is more basic than asking whether a simulated outcome is clinically plausible: first ask whether it is possible at all.
For example, a count of adverse events cannot be negative, a percentage defined on a zero-to-100 scale cannot be 113, and a trial that guarantees exactly 20 participants per arm cannot have a replicate with 21 in one arm and 19 in the other. A prior predictive simulation that produces such a value points to a mismatch between the model and the data-generating constraints, or to an error in the simulation or data representation.
The earlier material on prior predictive distributions and simulating trial replicates supplies the basic simulation framework. Here the focus is the support of the generated data: the set of outcomes the endpoint definition and design permit.
Support and Forbidden Outcomes
For a discrete outcome \(Y\), the support check asks whether \(P(Y=y\mid\theta)\) is zero whenever \(y\notin D\), for every allowed parameter value \(\theta\). For a continuous outcome, individual values have probability zero even when they are possible, so the corresponding question is whether every forbidden set \(A\) has \(P(Y\in A\mid\theta)=0\). The prior predictive distribution averages the sampling distribution over the prior \(\pi(\theta)\):
If the sampling model assigns probability zero to every forbidden outcome for every allowed \(\theta\), then the prior predictive distribution does too. For a discrete forbidden outcome \(y\), \(p(y\mid\theta)=0\) throughout the integral, so \(m(y)=\int 0\cdot\pi(\theta)\,d\theta=0\). For a forbidden set \(A\), the same argument applies to \(P(Y\in A\mid\theta)\) inside the integral. Consequently, a simulated forbidden value is evidence that the model, its implementation, or the stated constraints do not agree.
Support is not the same as plausibility. A feasible outcome may be very unlikely under the prior predictive distribution; that alone does not make it impossible. Conversely, if an outcome is structurally forbidden, even a small simulated probability is a model problem, not simply an uncomfortable tail.
Worked Checks for Endpoint Constraints
Worked Example: Negative Event Counts Under a Normal Model
Hypothetical setup. A trial plans to record the number \(Y\) of events in one group. Counts must be whole numbers from 0 through 20. Suppose a proposed prior predictive approximation is \(Y\sim N(3,2^2)\). Check the probability that it generates a negative value.
For this normal distribution, standardize the boundary at zero:
Thus, about 6.68% of draws fall below zero. A negative event count is impossible, so the approximation fails the support check. It also produces non-integer values with positive probability, which are not valid recorded counts. A normal approximation might be useful for some calculations, but using it as the generative distribution for the recorded count does not preserve the count’s support. The model should instead use a count-valued sampling distribution whose outcomes respect the trial’s limits.
Worked Example: A Measurement With Fixed Bounds
Hypothetical setup. An assay reports a percentage that, by definition and data-processing rules, must lie between 0 and 100 inclusive. A proposed prior predictive model is \(Y\sim N(96,8^2)\), where values are measured in percentage points. Find the probability of exceeding the upper bound.
Standardizing 100 gives \(z=(100-96)/8=0.5\). Therefore,
The model assigns about 30.85% probability to values above the defined maximum. At the lower boundary, \(P(Y<0)=\Phi((0-96)/8)=\Phi(-12)\), which is approximately \(1.8\times10^{-33}\). Even though lower-bound violations are negligible here, the substantial upper-bound probability makes this normal model inconsistent with the stated measurement definition. Possible revisions include selecting a distribution with bounded support or modelling an appropriate transformation of the percentage. The choice must reflect how the measurement is defined and recorded; changing the distribution should not be a cosmetic way to hide a genuine feature of the data.
Worked Checks for Design Constraints
Worked Example: Fixed Allocation in a Trial Replicate
Hypothetical setup. A two-arm trial uses a randomization scheme that guarantees exactly 20 participants in each arm. A simulation program instead assigns each of the 40 participants independently with probability 0.5 and generates 100 replicate allocations. Six replicates have allocations other than 20 and 20: four are 21 and 19, and two are 22 and 18. Summarize the support violation.
The design permits only the allocation \((20,20)\), so each of the six imbalanced replicates violates a design constraint. The observed fraction is \(6/100=0.0600\), or 6.00%. A rough Monte Carlo standard error for this simulated fraction is
This Monte Carlo error describes uncertainty in estimating the violation frequency under the simulation program; it does not make the violations compatible with the actual design. If the trial truly guarantees 20 participants per arm, the simulator should represent that design and generate only \((20,20)\). If allocation imbalance is permitted by the real randomization procedure, then the declared constraint was wrong and the valid allocation space must be specified accordingly.
A Practical Checking Workflow
Write down constraints before inspecting simulated replicates. This makes the check auditable and reduces the risk of deciding after the fact that an inconvenient outcome is “close enough.” The constraints should follow from the endpoint’s recorded form and the actual design, including any restrictions on enrollment, allocation, or follow-up that determine which data sets can occur.
State bounds, permitted values, units, and design restrictions explicitly. For an event count among \(n\) participants, for instance, the recorded count must be an integer from 0 through \(n\).
Ask what values the sampling distribution and the simulator can produce. Check that parameter values also lie in their permitted domain, since invalid parameters can create invalid draws or computation failures.
Tally each type of violation rather than relying only on averages, histograms, or a visual impression. Record the number of replicates checked and the constraints tested.
Check whether the issue lies in the model family, its parameterization, the implementation, or the statement of the design. Revise the source of the mismatch and rerun the check.
Diagnostics, Alternatives, and Limitations
A useful diagnostic report states each constraint, the number of violating replicates, and the fraction of simulations that violated it. If no violations appear in a finite simulation, that is not proof that the probability is exactly zero. It means only that none appeared in the simulations run; rare violations may require more draws or an analytic check. When violations are observed, the Monte Carlo fraction helps describe how often the simulator produces them, but a low fraction does not make a structurally impossible outcome acceptable.
Keep support checking distinct from two related activities. Inspecting the prior on parameters can reveal parameter values that are inadmissible, but valid parameter values do not guarantee valid simulated data. A posterior predictive check asks whether data generated after conditioning on observed data reproduce relevant features of those observations; it does not replace checking that the prior predictive model respects basic constraints before data are observed. The present check asks whether the data-generating model can produce values that the endpoint and design forbid.
The right revision depends on why the constraint was violated. A count model should generate valid counts rather than negative or fractional observations. A model for a bounded measurement should reflect whether the recorded quantity truly has hard bounds. A simulation should encode design restrictions that the real trial guarantees. Truncating a distribution can be appropriate when the truncated sampling model is scientifically and operationally justified, but truncation changes the model and its probabilities. It should be specified deliberately, not applied only to simulated draws after seeing inconvenient outcomes.
Constraints also need careful definition. A measurement described by a zero-to-100 scale may have recorded values limited to that interval, while a different raw instrument reading might exceed its nominal range because of noise or device behavior. Rounding, censoring, and data-processing rules can likewise change which recorded values are possible. If the rule is uncertain, document the uncertainty and clarify what quantity the model represents before labelling an outcome impossible.
Common Mistake
A support check is a basic but consequential model check: it can uncover an unsuitable distribution, an incorrectly encoded design, or a mismatch between the modelled quantity and the recorded endpoint. It cannot establish that all feasible predictions are clinically credible. Once forbidden outcomes are eliminated, plausible ranges and other features still need substantive review.
Check Your Understanding
Work through each question, then compare your reasoning with the solutions.
- A recorded count must be an integer from 0 to 12. A proposed predictive model assigns positive probability to \(Y=-1\). Is this a support violation? Solution: Yes. The valid data space excludes negative counts, so any positive probability at \(Y=-1\) violates the stated constraint.
- A normal predictive model has mean 50 and standard deviation 5 for a measurement defined to lie between 0 and 100. Calculate the probability that a draw exceeds 100. Solution: \(z=(100-50)/5=10\), so \(P(Y>100)=1-\Phi(10)\), approximately \(7.62\times10^{-24}\). This is effectively zero numerically, but the unbounded normal model still has positive probability above 100 and therefore does not have strictly bounded support.
- A trial design permits either 15:15 or 16:14 allocation, but a simulated replicate has allocation 17:13. What should be checked? Solution: The replicate violates the stated design constraint. Check that the simulator implements the allocation scheme correctly and that the permitted allocation set was specified accurately; do not treat 17:13 as acceptable merely because the total remains 30.
- What is the difference between a feasible but unlikely outcome and a support violation? Solution: A feasible outcome belongs to the valid data space and may have low probability under the model. A support violation lies outside that space but is assigned positive probability by the model.