Variance of a Treatment-Control Difference

Why Separate the Two Contributions?

In a multi-arm trial, each treatment is compared with a control group. For one such comparison, the estimated difference in means depends on observations from both arms. Its precision therefore reflects two sources of sampling variation: variation in the treatment-arm mean and variation in the control-arm mean. Keeping those contributions separate makes the variance calculation transparent and helps identify how sample sizes and outcome variability affect precision.

The focus here is one treatment-control comparison. A shared control also makes different treatment-control estimates statistically related, but calculating their covariance is a separate question. The earlier material on the sampling structure of shared-control comparisons establishes that a treatment arm and the control arm contain distinct observations, while the same control observations may be used in more than one comparison.

For the variance formula below, assume observations are independent within and between the two arms, and that each arm's observations have a common finite variance within that arm. The treatment and control variances need not be equal. Randomization is a common way to create independent arms in a parallel-group trial, but the variance calculation itself depends on the sampling structure and assumptions, not on randomization alone. No normality assumption is needed to calculate the variance. Normal outcomes do give an exact normal distribution for the difference in sample means.

Deriving the Variance

The variance of a difference contains a covariance term in general. Here, the treatment-arm sample mean and control-arm sample mean are independent, so that covariance is zero. Also, the variance of a mean of \(n\) independent observations with common variance \(\sigma^2\) is \(\sigma^2/n\). Applying those facts gives:

The first term is the treatment-arm contribution; the second is the control-arm contribution. Each is nonnegative, so both arms add sampling variation. Increasing one arm's sample size reduces that arm's contribution, but does not eliminate the contribution from the other arm.

If both arms have a common variance \(\sigma^2\), the formula simplifies to \(\sigma^2(1/n_i+1/n_0)\). This familiar special case still contains two contributions. It is not \(\sigma^2/(n_i+n_0)\): the treatment and control means are estimated separately and then subtracted.

Worked Calculations

The unpooled estimated standard error in the second example is often called the unequal-variance or Welch standard error. An alternative is the pooled standard error, which combines the two sample variances before calculating the standard error. That approach requires a common-variance model. Under that model, the pooled variance estimate is

For the hypothetical data above, \(s_p^2=[35(144)+63(81)]/98=103.5\) minutes squared, giving \(SE_{\mathrm{pooled}}=\sqrt{103.5(1/36+1/64)}=2.119\) minutes. This differs from 2.295 minutes because it imposes a common-variance assumption and weights the two sample variances together. It is not a more accurate standard error merely because it is smaller. When unequal variances are plausible, the unpooled calculation retains each arm's separate contribution.

Conditions, Diagnostics, and Alternatives

The formula is exact for the variance of the difference under independent observations with finite arm-specific variances. Its usual estimated version substitutes \(s_i^2\) and \(s_0^2\) for the unknown variances. The standard error is then an estimate of the standard deviation of the estimator, not the standard deviation of individual outcomes. If outcomes are normally distributed within each arm, the difference in sample means is normally distributed when population variances are known. With estimated variances, inference also depends on the chosen method and its assumptions; calculating this standard error alone does not determine a test or confidence interval.

Before using the calculation, check how observations were collected and whether the assumptions match the design. In particular, repeated measurements on one participant, clustering by site, or other within-arm dependence can make \(s^2/n\) an unsuitable variance estimate. In a paired design, the two measurements are not independent and their covariance matters. The independent-arm formula should not be applied without accounting for that dependence. Also check that each sample standard deviation and sample size refers to the same analysis population; missing outcomes or exclusions can change both.

For one comparison, the unpooled formula is the direct plug-in estimate when the two arm variances may differ. The pooled alternative is appropriate only when a common variance is a defensible assumption. A model-based analysis may instead adjust for prespecified covariates or reflect clustering, but its standard error must follow that model and design. These approaches answer related precision questions under different assumptions; they should not be interchanged without justification.

Common Mistake and Key Limitation

A common mistake is to use only the treatment-arm term, \(\sigma_i^2/n_i\), because the treatment is the arm of interest. That leaves out the sampling variation in the estimated control mean. The correction is to include both independent contributions: \(\sigma_i^2/n_i+\sigma_0^2/n_0\). Another error is to label this single-comparison variance as the variance of all treatment-control estimates jointly. It describes one contrast only; sharing a control across comparisons affects their joint dependence but does not remove the control contribution from any individual contrast.

The calculation also has practical limitations. A small standard error does not establish that the treatment effect is important, unbiased, or clinically meaningful. It quantifies sampling precision under the stated model. Poor measurement, selection, nonadherence, or a mismatch between the analysis population and the target population are not repaired by computing the formula correctly.