Variance Reduction With Common Random Numbers in Bayesian Trials

Why Pair Assurance Simulations?

A design decision often depends on comparing assurance under two candidate designs: for example, a total sample size of 160 versus 200, or a 1:1 treatment-control allocation versus a 2:1 allocation. Assurance is the prior-averaged probability of satisfying the specified success rule, so it can be estimated by simulating the design parameters and trial data, then recording whether each simulated trial succeeds. When two assurances are estimated separately, random variation in both estimates contributes to the uncertainty in their difference.

Common random numbers (CRN) aim to reduce that uncertainty by matching simulation replicates across the two designs. Each matched pair uses the same draws for shared sources of randomness, such as the design-prior parameters, while each design is still evaluated using its own sample size, allocation, analysis, and success rule. The comparison then depends on the variation in the paired difference, rather than on two unrelated sets of simulated outcomes.

This is a simulation technique, not a change to the trial model or to the definition of assurance. It is useful when the target is a contrast between designs. It does not necessarily improve the precision of either assurance estimate considered on its own.

Paired Estimation and Its Variance

The crucial requirement is that matching must not change either design’s marginal distribution. For example, using the same draw of a treatment-effect parameter for both designs is valid when that parameter is drawn from the same design prior in both simulations. The designs can then use that shared draw but apply their respective sample sizes, allocation ratios, likelihoods, and Bayesian success criteria. A coupling is simply a way of generating the two design outcomes together; it must leave each design’s standalone simulation valid.

Suppose the \(M\) matched pairs are independent and identically distributed. The variance of one paired difference follows from the variance formula for a difference:

The paired estimator averages \(M\) independent differences, so its variance is the variance above divided by \(M\). Positive covariance reduces the variance; negative covariance increases it. Since each outcome is an indicator, \(\operatorname{Var}(Y_A)=q_A(1-q_A)\) and \(\operatorname{Var}(Y_B)=q_B(1-q_B)\). The covariance is determined by the chosen coupling, not by the two assurances alone.

In practice, calculate \(D_m=Y_{Am}-Y_{Bm}\) for each pair. The Monte Carlo standard error (MCSE) of \(\hat{\Delta}\) is estimated by \(s_D/\sqrt{M}\), where \(s_D\) is the sample standard deviation of the \(M\) paired differences. This standard error quantifies simulation error in the estimated assurance difference. It is not a posterior standard deviation for a treatment effect, nor does it describe uncertainty about the clinical model or design prior.

The logic of the variance benefit is immediate from the formula: a coupling with positive covariance subtracts a positive quantity from the variance of the difference. In contrast, calculating two separate MCSEs and treating the estimates as independent omits the covariance term. That calculation can overstate or understate the actual simulation error of the paired contrast.

Constructing a Valid Coupling

For different sample sizes, a natural coupling is to simulate a sufficiently long sequence of potential observations for each arm and let each design use the number it requires. For different allocation ratios, use shared treatment-arm draws across designs and shared control-arm draws across designs, with the number used from each arm determined by each allocation. Use independent random streams for treatment and control within a simulated trial if the assumed data-generating model makes the arms independent. The shared streams are across candidate designs, not a reason to make the two arms dependent within a trial.

For binary outcomes, a shared uniform draw provides a simple construction. If an outcome has probability \(p\), generate \(U\sim\operatorname{Uniform}(0,1)\) and set the outcome to 1 when \(U<p\), and to 0 otherwise. For two designs with probabilities \(p_A\) and \(p_B\), using the same \(U\) for corresponding observations preserves each Bernoulli marginal. When \(p_A\) and \(p_B\) are close, the outcomes often agree, which can create positive covariance.

At the outer level of an assurance simulation, the same design-prior draw and any shared nuisance-parameter draws can be used for both designs. At the inner level, use a coupling that preserves the data-generating distribution under each design. Each simulated trial must then be analyzed under its own design and evaluated against its own specified success rule. A shared random number cannot justify applying one design’s posterior calculation or decision threshold to the other.

Worked Assurance Comparisons

Diagnostics, Alternatives, and Limitations

Common random numbers should be checked empirically rather than assumed to help. Inspect the frequencies of the four paired outcomes: success under both, failure under both, A-only success, and B-only success. Many matching outcomes often indicate positive association, but the decisive diagnostic is the observed variance of \(D_m\), or its comparison with the variance expected from independent simulation. Report the paired MCSE for the contrast and retain the two marginal assurance estimates so that the comparison can be audited.

A coupling can produce negative covariance. For example, imagine 1,000 pairs where each design succeeds in 500 replicates, but 450 pairs are A-only successes and 450 are B-only successes; 50 pairs succeed under both and 50 fail under both. Each estimated assurance is 0.500, but the paired difference is nonzero in 900 replicates. Its sample variance is \(900/999=0.9009\), giving an MCSE of \(\sqrt{0.9009/1000}=0.03002\). The independent-estimate MCSE would be \(\sqrt{[0.25+0.25]/1000}=0.02236\). Pairing has made this contrast less precise, not more precise.

Independent simulation is the main alternative: simulate each candidate design separately and estimate its assurance and MCSE. It is simple, but a contrast based on independent runs has no matched covariance benefit. Exact or deterministic calculation is another alternative when the design and model make it feasible; simulation is then not necessary for that calculation. The earlier material on Monte Carlo error for assurance estimates discusses simulation error for assurance estimates themselves; for a design contrast, the paired differences supply the relevant simulation-error calculation.

Several failure modes deserve attention:

• Changing a marginal distribution. Reusing draws is not valid if it gives either candidate design the wrong prior, likelihood, sample size, or outcome distribution. Verify each design’s marginal simulation recipe independently.
• Sharing draws across independent trial arms. If treatment and control outcomes are assumed independent conditional on parameters, use independent draws between arms within each replicate. Match treatment-arm draws across designs and control-arm draws across designs instead.
• Using separate estimates but a paired standard error. The paired MCSE requires a stored, correctly aligned \(D_m\) for every replicate. Marginal success totals alone do not reveal the covariance.
• Assuming CRN always improves precision. The sign and size of the covariance depend on the coupling and the success rules. Compare the paired-difference variability with the independent benchmark.
• Overinterpreting a small estimated difference. A difference may be small relative to its MCSE. Increase the number of paired replicates if simulation error is too large for the design decision, and do not confuse Monte Carlo precision with uncertainty from the prior or model assumptions.

The method is especially natural for nearby sample sizes or allocations, because shared parameter draws and nested observations can make simulated performance indicators move together. However, the success indicator is a thresholded result of a full Bayesian analysis; nearby designs do not guarantee a high correlation between indicators. The benefit must be judged from the paired simulation output.