Why Shared Controls Must Be Simulated Once
When several treatments are compared with one control group, each treatment-control contrast contains the same estimate of the control response. The contrasts are therefore generally dependent, even when participants in different trial arms have independent outcomes. A simulation that generates a fresh control sample for each contrast removes this shared component and can misrepresent the joint behavior of the comparisons.
The essential implementation rule is simple: within each simulated replicate, generate the control data once, then use that same control data in every treatment-control contrast. This tutorial derives the resulting dependence, shows how to reproduce it in simulation, and explains how to check that the simulation is behaving as intended.
Model and Dependence From a Shared Control
Consider one analysis stage with a control arm indexed by 0 and \(K\) treatment arms indexed by \(j=1,\ldots,K\). Let \(Y_{0i}\) denote the outcome for control participant \(i\), and \(Y_{ji}\) the outcome for participant \(i\) in treatment arm \(j\). Assume independent participants and independent randomized arms. Within each arm, outcomes have mean \(\mu_j\) and variance \(\sigma_j^2\); the control outcomes have mean \(\mu_0\) and variance \(\sigma_0^2\). For the standard parallel-group model, treatment-arm outcomes and control outcomes are independent. The sample sizes are \(n_0,n_1,\ldots,n_K\).
Define the estimated treatment effect for arm \(j\) as the treatment sample mean minus the shared control sample mean. The dependence of interest concerns these estimated contrasts, not dependence between individual outcomes in different arms.
Because treatment arms are independent of each other and of the control arm, the covariance between two distinct contrasts \(j\) and \(k\) is
The first three terms in the expansion are zero under the independent-arm model. The remaining term is positive: both contrasts move in the same direction when the sampled control mean moves. The variance of a single contrast is
Thus the correlation between two distinct contrasts is the shared-control variance divided by the square root of the product of their variances. This derivation does not require normally distributed outcomes, although normal outcomes make the vector of contrasts jointly normal.
Worked Example: Theoretical Correlation With Equal Allocation
Suppose a hypothetical trial has two treatment arms and one control arm, with 40 participants per arm. Assume independent normal outcomes with a common standard deviation of 10 outcome units in all arms. Find the variance of each treatment-control contrast, their covariance, and their correlation.
The outcome variance is \(10^2=100\) squared outcome units. The variance of either contrast is
The covariance between the two contrasts is the variance of the shared control mean:
Therefore, their correlation is
The positive correlation is not evidence that the treatment outcomes are associated with each other. It arises because the same estimated control mean is subtracted from both treatment means.
Generating a Replicate Correctly
A replicate represents one complete hypothetical trial under specified sample sizes, outcome distributions, and treatment effects. For a single analysis stage, first generate the control observations. Then generate each treatment arm independently, and calculate every contrast using the stored control sample mean. Repeat this entire process independently for the next replicate. The control data are shared within a replicate, not carried over between replicates.
Specify the arm means, variances, sample sizes, and outcome model. State whether the calculation is for a null or alternative treatment-effect configuration.
Draw \(n_0\) control outcomes from the specified control distribution and calculate \(\bar Y_0\) once.
Draw each treatment arm's \(n_j\) outcomes from its specified distribution, independently across arms under the model, and calculate each \(\bar Y_j\).
For every treatment arm, calculate \(\hat\theta_j=\bar Y_j-\bar Y_0\), reusing the same stored \(\bar Y_0\).
Store the complete vector \((\hat\theta_1,\ldots,\hat\theta_K)\). For the next replicate, generate a new control sample and new treatment samples.
In software, the crucial distinction is between one control draw per replicate and one control draw per contrast. A sound implementation has a control-generation step outside the loop over treatment contrasts but inside the loop over replicates. Treatment outcomes are generated separately for each arm. If the analysis uses a statistic more complicated than a difference in means, the same design principle still applies: form every comparison using the same replicate-level control data.
Worked Example: Binary Outcomes and Shared Dependence
For a hypothetical binary-outcome trial, suppose the control response probability is \(p_0=0.40\), and both treatment arms have response probability \(0.40\) under the global null. Let each arm have 30 participants. Within a replicate, draw one control sample and calculate its response proportion \(\hat p_0\); draw one sample for each treatment and calculate \(\hat p_1\) and \(\hat p_2\). The two estimated risk differences are \(\hat p_1-\hat p_0\) and \(\hat p_2-\hat p_0\).
Under independent Bernoulli sampling across arms, the control proportion has variance \(p_0(1-p_0)/n_0\), and the treatment proportions are independent of each other and of it. Hence
Each risk-difference estimator has variance \(0.24/30+0.24/30=0.0160\), so their correlation is \(0.00800/\sqrt{0.0160\times0.0160}=0.500\). The contrasts are dependent even though all three arms were generated independently. The dependence is created by reusing the same control sample proportion.
If a simulation instead draws an independent control sample for each treatment contrast, the two risk differences have no shared control term and their covariance is zero under this model. That is a different simulated design, not a harmless coding variation.
Checking the Simulated Dependence
The covariance formula gives a direct diagnostic for a simulation. Across \(R\) independent replicates, store each contrast as one column of a data matrix. The sample covariance between columns \(j\) and \(k\) estimates their theoretical covariance. Under the equal-variance normal model, the expected covariance is \(\sigma_0^2/n_0\). A discrepancy in a finite simulation is expected; the relevant question is whether it is compatible with Monte Carlo variability and whether the code uses the intended control sample.
The next small hypothetical output illustrates the arithmetic. Suppose four replicates produce the following rounded contrasts, in outcome units.
| Replicate | \(\hat\theta_1\) | \(\hat\theta_2\) |
|---|---|---|
| 1 | 0.100 | 0.050 |
| 2 | 0.120 | 0.065 |
| 3 | 0.080 | 0.035 |
| 4 | 0.110 | 0.055 |
Worked Example: Computing an Empirical Covariance
For the four hypothetical replicates above, the contrast means are \(\bar\theta_1=0.1025\) and \(\bar\theta_2=0.05125\) outcome units. The centered values are \((-0.0025,0.0175,-0.0225,0.0075)\) and \((-0.00125,0.01375,-0.01625,0.00375)\), respectively. The sample covariance uses the divisor \(R-1=3\):
For context, the corresponding sample variances are \(s_{11}=0.000875/3=0.0002917\) and \(s_{22}=0.00046875/3=0.0001563\), in squared outcome units. The empirical correlation is therefore approximately \(0.0002125/\sqrt{0.0002917\times0.0001563}=0.995\). This very high correlation is a feature of these four illustrative values, not a reliable estimate of a population correlation: four replicates are far too few for a stable assessment. With many replicates, compare estimated covariance and correlation with their model-based values, allowing for Monte Carlo variation.
A useful diagnostic is to check several things together: the replicate-level control sample size, the number of control draws per replicate, the empirical mean and variance in each arm, and the covariance matrix of the stored contrasts. If the control mean is accidentally recomputed inside each contrast calculation, the covariance may be too small. If the same control sample is mistakenly carried across replicates, the replicates are no longer independent and the apparent Monte Carlo variability can be distorted.
Alternatives, Limitations, and Common Mistakes
The main alternative is to simulate each treatment-control contrast separately, using newly generated control data for each comparison. That approach may be appropriate if the intended design truly has separate control groups for separate comparisons. It is not appropriate for a design with one shared control group: it changes the joint distribution and sets the shared-control contribution to the covariance to zero under the independent-arm model. Another valid approach is to simulate the vector of contrasts directly from its joint distribution, provided its means and full covariance matrix are specified correctly. Direct multivariate simulation can be efficient for a known normal model, but it does not create individual participant data and is less convenient when outcomes are binary or the simulation needs additional participant-level quantities.
The formulas above assume independent arms, independent participants, and a correctly specified variance for the control outcomes. Unequal treatment-arm variances change the marginal contrast variances, but not the shared-control covariance \(\sigma_0^2/n_0\) under these assumptions. Other designs, such as paired outcomes or correlated observations within sites, require their own covariance structure. The simple formula should not be used without checking that the sampling model matches the design.
- Wrong reasoning: “Treatment arms are randomized independently, so their treatment-control contrasts are independent.” Correction: Each contrast also contains the same random control mean; under independent arms their covariance is \(\operatorname{Var}(\bar Y_0)\).
- Wrong implementation: Draw a new control sample for every contrast within a replicate. Correction: Generate the control data once per replicate and reuse them for all contrasts.
- Wrong diagnostic: Expect the empirical covariance from a small number of replicates to equal the theoretical covariance exactly. Correction: Sampling variation remains, so use enough replicates and assess discrepancies in light of Monte Carlo variability.
Check Your Understanding
Try these questions, then compare your reasoning with the solutions.
- Under independent normal outcomes with common variance \(\sigma^2\), write the covariance of two distinct contrasts when the shared control sample size is \(n_0\).
- With control variance 64, control sample size 32, and treatment sample sizes 32, what is the covariance and correlation of two contrasts if both treatment variances are also 64?
- In a binary-outcome simulation with control probability \(p_0\), what shared-control term contributes to the covariance of two risk-difference estimators?
- Why should a simulation generate a new control sample for each replicate rather than reusing one control sample for all replicates?
Solutions. 1. The covariance is \(\sigma^2/n_0\), because the shared control mean is the only random term common to the two contrasts. 2. The covariance is \(64/32=2.00\) squared outcome units. Each contrast variance is \(64/32+64/32=4.00\), so the correlation is \(2/\sqrt{4\times4}=0.500\). 3. The covariance contribution is \(\operatorname{Var}(\hat p_0)=p_0(1-p_0)/n_0\), under independent sampling across arms. 4. Replicates represent independent hypothetical trials; reusing the control sample would make their results dependent and misstate simulation variability.