Covariance of Standardized Treatment Statistics

Why Standardization Does Not Remove Shared-Control Dependence

Treatment-control contrasts are often put on a common scale by dividing each contrast by its standard error. This makes each statistic have variance one under the model, but it does not make statistics independent. When two treatment arms use the same control group, both contrasts contain the same random control mean, with a minus sign. That shared source of sampling variation remains after standardization.

The aim here is to derive the covariance and correlation for two such standardized contrasts under a common-variance model. The result is a pairwise statement: it concerns two treatment arms and their common control, not the construction or analysis of a full covariance matrix.

Model and Definition

Consider treatment arms $i$ and $j$, and a control arm indexed by $0$. Let arm $g$ have sample size $n_g$, population mean $\mu_g$, and independent observations with variance $\sigma^2$. Assume observations are independent within and across arms, all groups have the same variance, and each sample size is positive. The variance $\sigma^2$ is treated as known when forming the standard errors below.

The treatment-control contrast for arm $i$ is $\bar{x}_i-\bar{x}_0$. Its model-based marginal standard error is $\sigma\sqrt{1/n_i+1/n_0}$. Define $Z_i$ by dividing the contrast by that standard error; similarly define $Z_j$. If the statistic is written for testing a nonzero null contrast, subtracting the null value first does not change any covariance, because subtracting constants does not change covariance.

Definition: For a treatment arm $i$ compared with a shared control arm $0$, the standardized contrast is $$ Z_i=\frac{\bar{x}_i-\bar{x}_0}{\sigma\sqrt{1/n_i+1/n_0}}. $$ It is standardized by its own marginal standard error. For two treatment arms $i\ne j$, their covariance and correlation are calculated across repeated samples under the stated sampling model.

The common-variance assumption gives $\operatorname{Var}(\bar{x}_g)=\sigma^2/n_g$. Arm means from distinct groups are independent, so $\operatorname{Cov}(\bar{x}_i,\bar{x}_j)=0$, and each treatment-arm mean is independent of the control mean. These facts are enough to derive the covariance; normality is not required for this moment calculation.

Deriving the Standardized Covariance

Write each contrast as the treatment mean minus the same control mean. On expanding the covariance, the two treatment means contribute zero covariance with one another and with the control mean. The remaining term is the covariance of the two negative control means. Because both signs are negative, their product is positive.

\begin{aligned} \operatorname{Cov}(\bar{x}_i-\bar{x}_0,\bar{x}_j-\bar{x}_0) &=\operatorname{Cov}(\bar{x}_i,\bar{x}_j) -\operatorname{Cov}(\bar{x}_i,\bar{x}_0)\\ &\quad-\operatorname{Cov}(\bar{x}_0,\bar{x}_j) +\operatorname{Var}(\bar{x}_0)\\ &=\frac{\sigma^2}{n_0}. \end{aligned}

Dividing a random variable by a constant divides its covariance by that constant; here both constants are the marginal standard errors. Consequently,

\begin{aligned} \operatorname{Cov}(Z_i,Z_j) &=\frac{\sigma^2/n_0} {\sigma\sqrt{1/n_i+1/n_0}\;\sigma\sqrt{1/n_j+1/n_0}}\\ &=\frac{1/n_0} {\sqrt{(1/n_i+1/n_0)(1/n_j+1/n_0)}}\\ &=\sqrt{\frac{n_i n_j}{(n_i+n_0)(n_j+n_0)}}. \end{aligned}

Each standardized contrast has variance one: its numerator has variance $\sigma^2(1/n_i+1/n_0)$, exactly the square of its denominator. Therefore covariance and correlation are equal for this pair:

Result: Under the independent, common-variance model with standard errors formed using $\sigma$, $$ \operatorname{Var}(Z_i)=\operatorname{Var}(Z_j)=1,\qquad \operatorname{Corr}(Z_i,Z_j)=\operatorname{Cov}(Z_i,Z_j) =\sqrt{\frac{n_i n_j}{(n_i+n_0)(n_j+n_0)}}. $$ The shared control produces a positive covariance. The formula depends on the three sample sizes, not on the population means.

The result remains true when treatment effects are nonzero. For example, $E(Z_i)=(\mu_i-\mu_0)/[\sigma\sqrt{1/n_i+1/n_0}]$, which need not be zero. Covariance measures the joint variation around the means of the statistics, so shifting either statistic by its nonzero expectation leaves the covariance unchanged.

Worked Example: Equal Allocation

Suppose a hypothetical trial assigns 50 participants to each of two treatment arms and 50 to their shared control. Assume the common outcome standard deviation is known to be 10 points. Find the marginal standard errors and the covariance and correlation of the two standardized contrasts.

For either treatment arm, the marginal standard error is $$ SE=10\sqrt{\frac{1}{50}+\frac{1}{50}} =10\sqrt{0.04}=2.00\text{ points}. $$ The covariance of the unstandardized contrasts is the control-mean variance: $$ \operatorname{Cov}(\bar{x}_i-\bar{x}_0,\bar{x}_j-\bar{x}_0) =\frac{10^2}{50}=2.00\text{ points}^2. $$ Dividing by the product of the two marginal standard errors gives $$ \operatorname{Cov}(Z_i,Z_j)=\frac{2.00}{(2.00)(2.00)}=0.500. $$ Both standardized statistics have variance one, so their correlation is also $0.500$. Equal group sizes do not make the comparisons independent: they still share the control mean.

Unequal Sample Sizes and Interpretation

With unequal allocation, each treatment statistic has a different marginal standard error, but the same covariance derivation applies. The correlation formula shows how the control sample size and both treatment sample sizes jointly determine the standardized dependence. It is not generally valid to divide the raw covariance by just one standard error, or to use the same standard error for both comparisons.

Worked Example: Unequal Allocation

Consider a hypothetical trial with $n_0=40$, $n_i=30$, and $n_j=60$. Suppose the common known standard deviation is 12 mg/dL. Calculate the covariance and correlation of the two standardized contrasts.

The unstandardized covariance is $$ \frac{\sigma^2}{n_0}=\frac{12^2}{40}=3.60\text{ (mg/dL)}^2. $$ The two marginal standard errors are $$ SE_i=12\sqrt{\frac{1}{30}+\frac{1}{40}} =12\sqrt{\frac{7}{120}}\approx2.898\text{ mg/dL}, $$ and $$ SE_j=12\sqrt{\frac{1}{60}+\frac{1}{40}} =12\sqrt{\frac{1}{24}}\approx2.449\text{ mg/dL}. $$ Thus the standardized covariance is $$ \frac{3.60}{(2.898)(2.449)}\approx0.507. $$ The standard errors are marginally unit-standardizing, so this is also the correlation. Direct substitution confirms the calculation: $$ \sqrt{\frac{(30)(60)}{(30+40)(60+40)}} =\sqrt{\frac{1800}{7000}}\approx0.507. $$ The positive value means that, across repeated samples, a control mean that is unusually high tends to make both treatment-minus-control contrasts unusually low.

Correlation describes a symmetric, unit-free measure of linear dependence; covariance retains the units of the standardized statistics, which are dimensionless here as well. The practical distinction is useful when checking calculations: first derive covariance on the contrast scale, then divide by both standard errors. The equality of covariance and correlation follows only because each statistic has variance one.

Worked Example: Treatment Effects Shift Means, Not Covariance

In another hypothetical setting, let $n_0=32$, $n_i=32$, and $n_j=64$, with known common standard deviation $\sigma=8$ outcome units. Suppose the population treatment effects relative to control are 4 units for arm $i$ and $-2$ units for arm $j$. Calculate the expected standardized statistics and their covariance and correlation.

The marginal standard errors are $$ SE_i=8\sqrt{\frac{1}{32}+\frac{1}{32}}=2.00, $$ and $$ SE_j=8\sqrt{\frac{1}{64}+\frac{1}{32}} =8\sqrt{\frac{3}{64}}=\sqrt{3}\approx1.732. $$ Therefore $$ E(Z_i)=\frac{4}{2.00}=2.00,\qquad E(Z_j)=\frac{-2}{1.732}\approx-1.155. $$ The covariance is determined by the shared control: $$ \operatorname{Cov}(Z_i,Z_j) =\frac{8^2/32}{(2.00)(1.732)} =\frac{2.00}{3.464}\approx0.577. $$ Using the sample-size form gives the same result: $$ \sqrt{\frac{(32)(64)}{(32+32)(64+32)}} =\sqrt{\frac{2048}{6144}}\approx0.577. $$ The expected statistics have opposite signs, but their covariance is positive. The signs of the treatment effects determine the means of the statistics, not the covariance of their sampling errors.

Assumptions, Diagnostics, and Alternatives

Before using the formula, check that the comparisons really use the same control observations, that observations in different arms are independent, and that the variance is common across arms. Also verify that each statistic is standardized by its own model-based marginal standard error. These are substantive conditions, not details that can be repaired by the algebra.

Separate control samples: if each treatment comparison uses a different independent control sample, there is no shared control mean to create this covariance. That setting is not described by the shared-control formula.
Unequal variances: the common-variance standard-error expression no longer applies. The control contribution to the raw covariance is then governed by the control variance, while the marginal variances also change.
Estimated standard errors: replacing $\sigma$ by a pooled estimate makes the denominators random. The derivation above is exact for the stated known-$\sigma$ standardization; it should not automatically be treated as an exact covariance formula for statistics with a shared estimated denominator.
Dependence within or across groups: correlated observations or overlapping participants can add covariance terms absent from the independent-arm model.

A useful diagnostic is to expand the two contrasts before applying any formula. If both contain $-\bar{x}_0$, the control term contributes $+\operatorname{Var}(\bar{x}_0)$. A result with a negative sign usually signals an error in expanding the covariance. Another check is scale: the raw covariance must have squared outcome units, whereas the covariance of standardized contrasts is dimensionless.

Common Mistake

A frequent error is to reason that the shared control appears with a minus sign in each contrast, so its covariance contribution must be negative. That treats the signs as if only one contrast contained the minus sign. In the covariance expansion, the product of the two coefficients is $(-1)(-1)=+1$. Thus the shared-control contribution is $+\operatorname{Var}(\bar{x}_0)$, and the resulting standardized covariance is positive under the stated model.

A second error is to conclude that marginal standardization makes the statistics independent. It only sets each marginal variance to one. Independence requires stronger conditions and does not follow from unit variance; shared-control contrasts remain correlated.

Key takeaway: Under independent groups with a common known variance, two treatment-control contrasts sharing control arm $0$ have raw covariance $\sigma^2/n_0$. After division by their respective marginal standard errors, their covariance and correlation are both $\sqrt{n_i n_j/[(n_i+n_0)(n_j+n_0)]}$.

Check Your Understanding

Work each problem using the assumptions stated in the question. Solutions follow each item.

For $n_0=25$, $n_i=25$, and $n_j=25$, find the correlation of the standardized contrasts.
Solution: $\sqrt{(25)(25)/[(25+25)(25+25)]}=\sqrt{625/2500}=0.500$.
For $n_0=20$, $n_i=20$, and $n_j=80$, calculate the correlation.
Solution: $\sqrt{(20)(80)/[(20+20)(80+20)]}=\sqrt{1600/4000}=\sqrt{0.4}\approx0.632$.
Why does a negative treatment effect in arm $j$ not make the shared-control covariance negative?
Solution: The covariance is determined by sampling variation around each statistic's expectation. Changing a treatment effect shifts $E(Z_j)$ but does not change covariance. The shared control contributes $\operatorname{Var}(\bar{x}_0)>0$ to the raw covariance.
A researcher uses the same control group but reports a negative covariance under the independent, common-variance model. What algebraic term should be checked?
Solution: Expand $\operatorname{Cov}(\bar{x}_i-\bar{x}_0,\bar{x}_j-\bar{x}_0)$. The control-control term is $+\operatorname{Var}(\bar{x}_0)$, because both coefficients are negative. The other cross-group terms are zero under independence.