Worked Example With Two Treatment Arms

Setting up the two-arm calculation

When two treatments are each compared with the same control group, the resulting estimated treatment effects share data. That shared control contributes to the uncertainty of both estimates. The goal here is to calculate the variance of each difference, their covariance, and their correlation in the balanced equal-variance case. The calculations concern repeated-sampling variation in the estimated differences, not the numerical similarity of the two observed treatment effects.

Model and assumptions: Let \(Y_{ij}\) be the outcome for participant \(i\) in group \(j\), where \(j=0\) denotes control and \(j=1,2\) denote the two treatment groups. Each group has the same sample size \(n>0\). Outcomes are independent across participants and groups, with \(E(Y_{ij})=\mu_j\) and \(\operatorname{Var}(Y_{ij})=\sigma^2\), where the common variance satisfies \(\sigma^2\geq 0\). Define the treatment-control differences \(D_1=\bar{Y}_1-\bar{Y}_0\) and \(D_2=\bar{Y}_2-\bar{Y}_0\), where \(\bar{Y}_j\) is the sample mean in group \(j\). No normality assumption is required for the variance and covariance calculations below.

The equal sample sizes and common variance are essential to this particular balanced calculation. Independence means that the random errors from different participants and groups have zero covariance. The sample means are therefore independent across groups under this model, and each has variance \(\sigma^2/n\). The shared-control means in \(D_1\) and \(D_2\), however, are the same random quantity; they are not independent copies.

Deriving the variances and covariance

The expectation of \(D_1\) is \(\mu_1-\mu_0\), and the expectation of \(D_2\) is \(\mu_2-\mu_0\). For either difference, the treatment-group and control-group sample means are independent. Thus the variance of a difference is the sum of the two sample-mean variances:

\operatorname{Var}(D_1) =\operatorname{Var}(\bar{Y}_1-\bar{Y}_0) =\frac{\sigma^2}{n}+\frac{\sigma^2}{n} =\frac{2\sigma^2}{n}.

The same reasoning gives \(\operatorname{Var}(D_2)=2\sigma^2/n\). For the covariance, expand both differences. The treatment-group means are independent of one another and of the control mean, so all cross-group covariance terms vanish. The control contribution has a positive sign in the covariance because it is subtracted in both differences.

\begin{aligned} \operatorname{Cov}(D_1,D_2) &=\operatorname{Cov}(\bar{Y}_1-\bar{Y}_0,\bar{Y}_2-\bar{Y}_0)\\ &=\operatorname{Cov}(\bar{Y}_1,\bar{Y}_2) -\operatorname{Cov}(\bar{Y}_1,\bar{Y}_0) -\operatorname{Cov}(\bar{Y}_0,\bar{Y}_2) +\operatorname{Var}(\bar{Y}_0)\\ &=0-0-0+\frac{\sigma^2}{n} =\frac{\sigma^2}{n}. \end{aligned}

When \(\sigma^2>0\), both differences have positive standard deviation, so their correlation is defined. Dividing their covariance by the product of their standard deviations gives:

\operatorname{Corr}(D_1,D_2) =\frac{\sigma^2/n} {\sqrt{(2\sigma^2/n)(2\sigma^2/n)}} =\frac{1}{2}.

Key result: With independent observations, equal group sizes \(n>0\), and a common variance \(\sigma^2>0\), each treatment-control difference has variance \(2\sigma^2/n\), their covariance is \(\sigma^2/n\), and their correlation is \(1/2\). If \(\sigma^2=0\), both difference variances and their covariance are zero, but the correlation is undefined because its denominator is zero.

This last condition matters: a formula for correlation requires nonzero standard deviations. The covariance formula itself remains valid at \(\sigma^2=0\), but it does not make the correlation equal to one-half in that degenerate case. The result also does not mean that the two estimated effects are numerically equal or that either is more likely to be positive. It describes how their sampling fluctuations move together under the stated model.

Worked calculations

Worked Example: Planning Variances With a Known Common Variance

Suppose a hypothetical balanced design has 40 participants in each of the control and two treatment groups. For planning, assume a common outcome variance of \(100\) squared outcome units. Calculate the variance, standard error, covariance, and correlation of the two treatment-control differences.

The groups have independent observations and the same variance, and each has \(n=40\). The assumed variance is positive, so both contrast standard deviations are nonzero and a correlation can be calculated.

For each difference, \(\operatorname{Var}(D_j)=2\sigma^2/n=2(100)/40=5.00\) squared outcome units. Its standard error is \(\sqrt{5.00}=2.236\) outcome units.

The covariance is \(\operatorname{Cov}(D_1,D_2)=\sigma^2/n=100/40=2.50\) squared outcome units. Therefore, \(\operatorname{Corr}(D_1,D_2)=2.50/\sqrt{(5.00)(5.00)}=0.500\).

The covariance is positive because an unusually high control mean lowers both treatment-minus-control estimates, while an unusually low control mean raises both. The correlation is dimensionless; the variance and covariance retain squared outcome units.

Worked Example: Plugging in an Estimated Common Variance

In a hypothetical study, each of three groups has 25 participants. The observed means are 10.2 units for control, 13.0 units for treatment 1, and 12.1 units for treatment 2. Suppose the pooled within-group variance estimate is \(s_p^2=64\) squared units. Calculate the observed differences and the covariance quantities estimated by substituting this common variance estimate.

The common sample size and pooled variance are being used under an equal-variance model; observations are assumed independent. The estimate \(s_p^2=64\) is positive, so the plug-in standard errors are nonzero. The observed treatment-control differences are \(d_1=13.0-10.2=2.80\) units and \(d_2=12.1-10.2=1.90\) units.

Substituting \(s_p^2\) for \(\sigma^2\), each estimated variance is \(2s_p^2/n=2(64)/25=5.12\) squared units. The estimated standard error of either difference is \(\sqrt{5.12}=2.263\) units. The estimated covariance is \(s_p^2/n=64/25=2.56\) squared units.

The resulting estimated correlation is \(2.56/\sqrt{(5.12)(5.12)}=0.500\). This plug-in correlation is fixed by the balanced equal-variance structure, not by whether the observed differences are 2.80 and 1.90 units or some other values. The differences describe estimated effects; their covariance describes the estimated effects’ sampling dependence.

Worked Example: Checking a Linear Combination of the Two Differences

Consider another hypothetical balanced design with \(n=16\) per group and common variance \(\sigma^2=36\) squared units. Suppose the group means are 20, 23, and 22 units for control, treatment 1, and treatment 2, respectively. Calculate the variance of the difference between the two treatment-control estimates, \(D_1-D_2\), and verify it directly from the group means.

The assumptions are independent observations, equal group sizes, and a positive common variance. The estimated treatment-control differences are \(d_1=23-20=3\) units and \(d_2=22-20=2\) units, so \(d_1-d_2=1\) unit.

Using the variances and covariance, \(\operatorname{Var}(D_1-D_2)=\operatorname{Var}(D_1)+\operatorname{Var}(D_2)-2\operatorname{Cov}(D_1,D_2)\). Here each variance is \(2(36)/16=4.50\) squared units, and the covariance is \(36/16=2.25\) squared units. Hence \(4.50+4.50-2(2.25)=4.50\) squared units, with standard error \(\sqrt{4.50}=2.121\) units.

Directly, \(D_1-D_2=(\bar{Y}_1-\bar{Y}_0)-(\bar{Y}_2-\bar{Y}_0)=\bar{Y}_1-\bar{Y}_2\); the shared control cancels. The two treatment means are independent, so the variance is \(36/16+36/16=4.50\) squared units, agreeing with the covariance calculation. This check makes clear why ignoring the covariance would give the wrong variance for a combination of contrasts.

Checking the result and understanding alternatives

The two contrast estimates have covariance matrix

\operatorname{Cov} \begin{pmatrix}D_1\\D_2\end{pmatrix} = \frac{\sigma^2}{n} \begin{pmatrix} 2 & 1\\ 1 & 2 \end{pmatrix}.

For \(\sigma^2>0\), this matrix has positive diagonal entries and determinant \(3(\sigma^2/n)^2>0\), so it is positive definite. Its diagonal entries reproduce the two variances, and its off-diagonal entries reproduce the shared-control covariance. As a practical check, covariance has squared outcome units, whereas correlation has no units and must lie between \(-1\) and \(1\). A result with correlation greater than one, negative covariance under this same-direction shared-control setup, or inconsistent diagonal entries signals a sign, scaling, or arithmetic error.

The main alternative design relevant to interpreting this dependence is to use separate control groups for the two comparisons. If those controls and treatment groups are independent, each treatment-control estimate uses a different control mean, and the model-based covariance between the two estimates is zero. That design changes the allocation of participants and the information available for each comparison; it is not a calculation that can be substituted while retaining the shared-control data. In the shared-control design, treating the two differences as independent is incorrect even though the treatment groups themselves are independent.

The correlation of one-half is specific to the assumptions here: three independent groups, one common control, equal group sizes, and a common positive variance. Unequal sample sizes or unequal variances change the calculation, so this numerical correlation should not be carried over to those settings. Nor does the model-based calculation capture every possible dependence arising from more complex data structures, such as correlated outcomes within participants. The assumed independence and variance structure should therefore be justified from the design and outcome model, rather than inferred from the two observed differences.

Common pitfalls and diagnostics

Dropping the shared-control term. The control mean appears in both differences. Its variance contributes \(+\sigma^2/n\) to their covariance because it has the same negative sign in each contrast.
Confusing covariance with correlation. The covariance is \(\sigma^2/n\) in squared outcome units. Only after dividing by both standard deviations does it become the unit-free correlation \(1/2\).
Claiming correlation when the variance is zero. If \(\sigma^2=0\), the standard deviations are zero, so the correlation is undefined. A ratio cannot be evaluated by cancelling terms that are all zero.
Using the balanced result outside its assumptions. Check that the sample sizes match, the group variances are modeled as equal, and observations are independent before applying the one-half result.
Using the observed effect values to infer their correlation. The correlation is a property of the joint sampling distribution under the model. Two observed values alone do not determine it.

Key takeaway: In the balanced equal-variance model with \(\sigma^2>0\), the shared control gives two treatment-control differences covariance \(\sigma^2/n\) and correlation \(1/2\). Verify the assumptions and keep the zero-variance case separate: there, correlation is undefined.

Check Your Understanding

Answer each question, then compare with the solutions below.

With \(n=20\) per group and \(\sigma^2=50\) squared units, calculate the variance of each treatment-control difference and their covariance.
In the same balanced model, what is the correlation when \(\sigma^2=0\)? Explain.
Why is the covariance positive when both differences subtract the same control mean?
If two otherwise independent treatment-control comparisons use separate independent control groups, what is their model-based covariance?

Solutions. (1) Each variance is \(2(50)/20=5.00\) squared units, and the covariance is \(50/20=2.50\) squared units. (2) It is undefined: both difference variances are zero, so the denominator in the correlation is zero. (3) The same control mean enters both differences with a negative sign; its contribution to their covariance is the product of those signs times its variance, \((-1)(-1)(\sigma^2/n)=\sigma^2/n>0\) when \(\sigma^2>0\). (4) Zero, provided the two comparisons use independent groups throughout, including their separate controls.