Stratified Randomization In Simulation Studies

Why Stratification Belongs in a Trial Simulation

A trial may stratify randomization by a baseline factor expected to be prognostic, such as disease severity or study site. Stratification fixes treatment counts within each stratum, preventing chance imbalances in those factors. A simulation study that evaluates this design should represent that assignment mechanism explicitly. Otherwise, the simulated trials do not correspond to the design whose operating characteristics are being assessed.

The useful comparison is usually not simply “stratified or not?” It is: under the same outcome-generating scenarios and the same decision rule, how do the operating characteristics differ when assignments are stratified rather than generated by the specified unstratified design? This comparison must preserve the trial’s intended treatment totals, estimand, analysis, and decision rule. A difference in simulated decision rates is meaningful only after those features have been made comparable.

Generating Assignments Correctly

For fixed-count stratified randomization, generate a separate random selection within each stratum. Select exactly \(n_{1h}\) of its \(N_h\) participants for treatment; assign the remainder to control. If all assignments of the required size are equally likely, each subset has probability \(1/{N_h \choose n_{1h}}\). If the strata are randomized independently, the probability of a particular complete allocation is the product of these stratum-specific probabilities.

The counts must match the design. With equal allocation, an even-sized stratum can be split equally; an odd-sized stratum requires a prespecified rule for rounding across strata or for handling the remaining participant. For unequal allocation, choose integer counts that implement the target ratio as closely as the design requires. Do not quietly alter those counts during simulation to make the results look balanced.

The unstratified comparison must also be specified precisely. For example, complete unstratified randomization might select exactly \(n_1\) of \(N\) participants for treatment, without fixing treatment counts by stratum. This is different from assigning each participant independently with probability \(1/2\), which allows the total treatment count to vary. Compare the stratified design with the unstratified mechanism that the trial would actually use as its alternative.

Simulating Operating Characteristics Under Both Designs

An operating characteristic is a property of a trial procedure under a specified outcome-generating scenario. Examples include the probability of crossing a prespecified decision threshold, the probability of declaring benefit under a null scenario, or the probability of reaching a target precision. For each simulated trial, generate the trial data according to the scenario, apply the prespecified analysis and decision rule, and record the result.

To compare assignment designs, hold the scenario and the decision procedure fixed wherever possible. The simulated stratum memberships, outcome-generating parameters, sample size, allocation ratio, analysis, and decision threshold should be aligned. Generate assignments according to each design. If the analysis is intended to account for strata, use that analysis consistently with the design; if the goal is to compare design-and-analysis strategies, define each strategy as a complete procedure and state that comparison explicitly.

When feasible, use matched simulation replicates: for replicate \(r\), use the same underlying scenario inputs for both assignment designs, then generate the two assignments and apply the decision rule separately. This common-random-number approach can reduce Monte Carlo noise in the difference because the replicate outcomes may be correlated. The assignments themselves must still be generated according to their respective randomization mechanisms. Pairing simulations does not mean reusing one design’s assignment for the other.

Estimating and Comparing Decision Rates

With \(R\) simulated trials, let \(D_r=1\) if replicate \(r\) meets the specified decision criterion and \(D_r=0\) otherwise. The estimated operating characteristic is \(\hat{\theta}=R^{-1}\sum_{r=1}^R D_r\). Because the replicates are simulated draws, its Monte Carlo standard error is estimated by \(\sqrt{\hat{\theta}(1-\hat{\theta})/R}\). This quantifies simulation error in estimating the scenario-specific probability; it is not the trial’s sampling standard error.

For matched comparisons, let \(D_{Sr}\) and \(D_{Ur}\) be the decision indicators in replicate \(r\) for stratified and unstratified assignment, respectively. Define \(Z_r=D_{Sr}-D_{Ur}\), which takes values \(-1,0,1\). The estimated difference in operating characteristics is \(\bar{Z}=R^{-1}\sum_r Z_r\), equal to the stratified estimated decision rate minus the unstratified estimated decision rate. Its Monte Carlo standard error is \(s_Z/\sqrt{R}\), where \(s_Z\) is the sample standard deviation of the replicate differences. Estimating the standard error from paired differences retains the correlation between methods within a replicate.

Justification for the Paired Monte Carlo Standard Error

Each \(Z_r\) is a replicate-level difference. If the matched replicates are independent draws from the simulation procedure, then the \(Z_r\) are independent draws from a distribution with variance \(\operatorname{Var}(Z_r)\). The average of \(R\) such draws has variance

Replacing the unknown replicate variance by \(s_Z^2\) gives the estimated Monte Carlo standard error \(s_Z/\sqrt{R}\). This differs from treating the two estimated rates as independent: the independent-rate calculation omits the covariance between the two outcomes within each matched replicate. The paired calculation is appropriate only when the replicates are genuinely matched as described and are independent across replicate index.

Diagnostics, Alternatives, and Limitations

Check each simulation run for implementation errors before interpreting its operating characteristics. For fixed-count stratified randomization, verify that every replicate has the required number assigned to each arm in every stratum, that no participant is assigned twice, and that the total allocation matches the design. Inspect stratum-specific event rates and decision rates as diagnostics, not as evidence by themselves that the assignment code is correct. Also confirm that the analysis uses the intended stratum information and that all scenario parameters are applied consistently across the design comparison.

Several alternatives answer different questions. Complete unstratified randomization preserves the overall arm totals but allows stratum counts to vary. Independent Bernoulli assignment allows both stratum and total counts to vary, so it is not a like-for-like substitute when the trial fixes its total allocation. Covariate-adaptive procedures may balance factors without using the same fixed-count mechanism; they should be simulated as their own design rather than labeled stratified randomization. The choice depends on the actual design being calibrated.

Stratification ensures balance in the factors used to define the strata, but it does not guarantee balance in other prognostic variables, eliminate chance variation within strata, or correct a misspecified outcome model. Very many or very small strata can make fixed-count allocation awkward, particularly when counts are rounded or a stratum has few participants. Treatment-effect heterogeneity also matters: a marginal decision rate or effect summary may depend on how stratum-specific results are weighted. State the estimand and weights rather than assuming that a single unweighted average answers every clinical question.

A frequent mistake is to compare two separately simulated rates and interpret their difference as a precise design effect without accounting for Monte Carlo uncertainty. Another is to use the paired standard error but pair unrelated replicates, or to reuse one assignment under both methods. The correction is to define the two randomization mechanisms, match only their underlying scenario inputs where appropriate, compute replicate-level differences, and estimate uncertainty from those differences. A large Monte Carlo standardized difference means the simulation estimate is stable relative to simulation noise; it does not remove uncertainty about whether the scenario adequately represents future trials.