Checking Virtual-Cohort Size and Monte Carlo Stability

Why Virtual-Cohort Size Needs to Be Checked

A virtual cohort is a finite random sample from a specified population model. Its phenotype proportions and simulated exposure quantiles will generally differ slightly from the corresponding model-population quantities. These differences are Monte Carlo error: they arise because a finite number of virtual subjects were sampled, not necessarily because the PBPK model or its input assumptions are wrong.

A cohort can be large enough to estimate a common phenotype proportion reasonably well but still be too small to estimate an exposure quantile in a sparse tail reliably. Cohort-size assessment should therefore be tied to the summaries that matter: for example, the precision required for a rare phenotype proportion and the acceptable uncertainty in a high exposure quantile. The calculations below assess sampling stability under the specified simulation design; they do not establish that the population assumptions or PBPK model are correct.

Planning for a Phenotype Proportion

Suppose each independently sampled subject either has a phenotype of interest or does not, and the model assigns probability \(p\) to that phenotype. If \(X_i=1\) denotes that subject \(i\) has the phenotype and \(X_i=0\) otherwise, then \(X_i\) is Bernoulli with probability \(p\). The cohort proportion is \(\hat p=\sum_{i=1}^{n}X_i/n\). Under independent sampling, its expected value is \(p\), and its standard error is \(\sqrt{p(1-p)/n}\).

For a planning calculation, choose a desired approximate 95% margin of error \(d\). Using the normal approximation with \(z=1.96\), the margin is approximately \(1.96\sqrt{p(1-p)/n}\). Solving for \(n\) gives the following planning formula. It assumes independent Bernoulli draws and uses a normal approximation; it is a design guide, not an exact guarantee of a particular interval width.

If \(p\) is not known, a defensible planning value should come from a prior estimate or a pilot simulation. Choosing \(p=0.5\) gives the largest value of \(p(1-p)\) and therefore a conservative size for a proportion under this formula. For a rare phenotype, using \(p=0.5\) may be unnecessarily cautious if a reliable rare-phenotype probability is available. Conversely, an inaccurate planning value can yield too small a cohort.

Checking Stability With Repeated Simulations

A direct diagnostic is to generate \(R\) independent virtual cohorts, each of size \(n\), using the same population-sampling design and PBPK model. Calculate the target summary separately for each cohort. The spread of those replicate estimates reveals how sensitive the summary is to finite-cohort sampling. Independence between replicates matters: reusing the same subjects, or using nearly identical random-number streams without appropriate design, can make the apparent spread misleadingly small.

For a phenotype proportion, the theoretical standard error \(\sqrt{p(1-p)/n}\) is a useful benchmark when \(p\) is known or well estimated. The sample standard deviation of replicate proportions also estimates their Monte Carlo spread, although it can be imprecise when \(R\) is small. Replicates should be numerous enough to make the stability assessment informative; a few similar estimates are not strong evidence of stability.

Why Exposure Quantiles Need a Separate Check

An exposure quantile is an order statistic: it depends on the ranked simulated exposures, rather than only on a binary indicator. For a continuous exposure distribution with cumulative distribution function \(F\), let \(q_{\tau}\) be the population quantile satisfying \(F(q_{\tau})=\tau\), and let \(\hat q_{\tau}\) be the corresponding empirical quantile from a cohort. Its sampling precision depends on both \(n\) and the density near \(q_{\tau}\). When the density is low near a tail quantile, a small change in rank can correspond to a substantial change in exposure.

A useful large-sample approximation follows by considering the number of cohort values at or below the true quantile. Under independent sampling from a continuous distribution, that count has a binomial distribution with probability \(\tau\). Its standard deviation is \(\sqrt{n\tau(1-\tau)}\), so the uncertain rank is approximately that many observations. Near \(q_{\tau}\), a change of one rank corresponds approximately to an exposure change of \(1/[nf(q_{\tau})]\), where \(f(q_{\tau})\) is the density at the quantile. Multiplying rank uncertainty by this local spacing gives the approximate standard error:

This approximation requires independent, identically distributed observations from a continuous distribution with a smooth, positive density near the target quantile, and a sufficiently large sample for the local approximation to be useful. The density has units of inverse exposure, so the resulting standard error has the same units as exposure. A very small or poorly estimated \(f(q_{\tau})\) makes the approximation unstable; repeated-cohort quantiles or order-statistic methods can then be more informative.

Practical Stability Assessment and Alternatives

A defensible assessment begins by identifying the summary and its required precision, rather than selecting a cohort size by habit. For proportions, use a binomial standard-error calculation to plan and then check with repeated independent cohorts. For quantiles, compare repeated-cohort estimates, examine the number of observations in the relevant tail, and consider an order-statistic interval or a bootstrap that resamples subjects in a way consistent with the cohort design. A bootstrap reflects finite-sample variability under the observed simulated cohort; it cannot correct a misspecified population distribution.

The simple binomial calculation is appropriate for independently sampled subjects with a common phenotype probability. It does not apply unchanged when subjects are deliberately sampled in fixed phenotype or genotype strata. Such designs can improve representation of rare groups, but the overall population proportion must then be estimated with the design weights, and its uncertainty must reflect the stratified sampling design. The earlier material on stratified virtual populations provides the relevant design context; a simple unweighted proportion from an intentionally balanced cohort generally does not estimate the population proportion.

For quantiles, repeated independent cohorts are often a transparent alternative to relying solely on the density-based approximation. If the replicate quantiles vary more than the required tolerance, increase cohort size and repeat the check. If the tail is sparse, also verify that the simulated exposure values and relevant ranks are plausible. Repeated simulations cannot diagnose every source of error: a stable result can still be consistently wrong because of misspecified allele frequencies, covariate dependence, genotype effects, physiology, or PBPK parameters.

Common Pitfalls and Limitations

• Using a single cohort as proof of stability. One estimate contains no direct information about how much it would change under another draw. Compare independent replicates or use a justified sampling-error calculation.
• Equating biological variability with Monte Carlo error. Variation among subjects is a feature of the population being simulated; Monte Carlo error is the uncertainty in a finite-cohort summary. A larger cohort reduces the latter, not the underlying biological variability.
• Applying the proportion formula to a quantile. Quantile precision depends on the density near the target quantile as well as on \(n\). A cohort adequate for a common phenotype proportion may be inadequate for a high exposure quantile.
• Treating correlated or stratified draws as independent, identically distributed draws. Dependence or deliberate allocation changes the variance calculation. Use a method that matches the sampling design.
• Interpreting Monte Carlo stability as model validation. Replicate agreement shows that sampling noise is limited for the chosen summary and design. It does not show that the simulated population or PBPK model represents the intended clinical population correctly.