Worked Example With Unequal Sample Sizes

Why Unequal Sample Sizes Matter

When several treatment groups are each compared with the same control group, the contrasts share the control-group mean. This creates dependence between the contrasts. If the treatment and control sample sizes differ, each contrast also has its own variance, so the correlations are not all the same. A reliable calculation starts with the group sample sizes and the shared-control structure, rather than treating the contrasts as independent.

This tutorial considers three treatment groups, indexed \(1,2,3\), and one control group, indexed \(0\). The outcome is measured on the same scale in every group. The main task is to calculate the covariance matrix and pairwise correlations of the three treatment-minus-control sample mean contrasts.

Definition: Let \(\bar{X}_i\) be the sample mean in group \(i\), based on \(n_i\) observations, and define the treatment-control contrast \(D_i=\bar{X}_i-\bar{X}_0\) for \(i=1,2,3\). The covariance matrix of the contrasts has entry \(\operatorname{Cov}(D_i,D_j)\) in row \(i\), column \(j\). The correlation is \(\operatorname{Corr}(D_i,D_j)=\operatorname{Cov}(D_i,D_j)/\sqrt{\operatorname{Var}(D_i)\operatorname{Var}(D_j)}\).

Model and Formula

Assume observations are independent within and across the four groups, each group has a finite outcome variance, and all groups share a common variance \(\sigma^2\). The group means therefore have variances \(\operatorname{Var}(\bar{X}_i)=\sigma^2/n_i\). The means in distinct groups are independent. Normal outcomes are not needed for these variance and covariance formulas; if outcomes are normally distributed, the vector of contrasts is jointly normal as well.

Each contrast subtracts the same control mean. That shared term contributes \(\sigma^2/n_0\) to the covariance of any two distinct contrasts. The treatment means themselves are from separate independent groups, so they contribute no cross-covariance.

\operatorname{Var}(D_i)=\sigma^2\left(\frac{1}{n_i}+\frac{1}{n_0}\right),\qquad \operatorname{Cov}(D_i,D_j)=\frac{\sigma^2}{n_0}\quad (i\ne j).

Thus, for the ordering \((D_1,D_2,D_3)\), the covariance matrix is

\boldsymbol{\Sigma}_D = \sigma^2 \begin{pmatrix} \frac{1}{n_1}+\frac{1}{n_0} & \frac{1}{n_0} & \frac{1}{n_0}\\ \frac{1}{n_0} & \frac{1}{n_2}+\frac{1}{n_0} & \frac{1}{n_0}\\ \frac{1}{n_0} & \frac{1}{n_0} & \frac{1}{n_3}+\frac{1}{n_0} \end{pmatrix}.

Dividing each off-diagonal covariance by the appropriate two standard deviations gives the pairwise correlations:

\rho_{ij} = \frac{1/n_0}{\sqrt{(1/n_i+1/n_0)(1/n_j+1/n_0)}} = \frac{\sqrt{n_i n_j}}{\sqrt{(n_0+n_i)(n_0+n_j)}},\qquad i\ne j.

Why the Shared Control Produces the Covariance

The covariance formula follows directly from expanding the contrasts. For distinct treatment groups \(i\) and \(j\),

\begin{aligned} \operatorname{Cov}(D_i,D_j) &=\operatorname{Cov}(\bar{X}_i-\bar{X}_0,\bar{X}_j-\bar{X}_0)\\ &=\operatorname{Cov}(\bar{X}_i,\bar{X}_j) -\operatorname{Cov}(\bar{X}_i,\bar{X}_0) -\operatorname{Cov}(\bar{X}_0,\bar{X}_j) +\operatorname{Var}(\bar{X}_0)\\ &=0-0-0+\frac{\sigma^2}{n_0} =\frac{\sigma^2}{n_0}. \end{aligned}

The three zero terms follow from independence of the group means. The final term is positive because the control mean is subtracted in both contrasts. If the control mean happens to be high, both treatment-minus-control contrasts tend to be lower; if it is low, both tend to be higher. This common source of variation induces positive covariance.

The diagonal entries follow from the variance of a difference of independent sample means: \(\operatorname{Var}(\bar{X}_i-\bar{X}_0)=\operatorname{Var}(\bar{X}_i)+\operatorname{Var}(\bar{X}_0)\). The sample sizes can differ freely in these expressions. They affect the diagonal entries separately, while the shared-control contribution to every off-diagonal entry is determined by \(n_0\).

Worked Example: Constructing the Matrix From Unequal Sample Sizes

Suppose a hypothetical experiment has \(n_0=40\), \(n_1=24\), \(n_2=36\), and \(n_3=60\). Assume independent groups with common variance \(\sigma^2=100\) outcome-units squared. Calculate the covariance matrix and the three pairwise correlations.

For \(D_1\), the variance is \(100(1/24+1/40)=100(0.06667)=6.667\). For \(D_2\), it is \(100(1/36+1/40)=100(0.05278)=5.278\). For \(D_3\), it is \(100(1/60+1/40)=100(0.04167)=4.167\). Each off-diagonal covariance is \(100/40=2.500\). Therefore,

\boldsymbol{\Sigma}_D= \begin{pmatrix} 6.667&2.500&2.500\\ 2.500&5.278&2.500\\ 2.500&2.500&4.167 \end{pmatrix}.

The pairwise correlations are obtained using the corresponding diagonal entries. For example, \(\rho_{12}=2.500/\sqrt{6.667(5.278)}=0.4215\). Likewise, \(\rho_{13}=2.500/\sqrt{6.667(4.167)}=0.4743\), and \(\rho_{23}=2.500/\sqrt{5.278(4.167)}=0.5331\). The correlation matrix is therefore

\mathbf{R}_D= \begin{pmatrix} 1&0.4215&0.4743\\ 0.4215&1&0.5331\\ 0.4743&0.5331&1 \end{pmatrix}.

The covariance entries have units of outcome-units squared, whereas correlations are dimensionless. The different correlation values arise because the treatment-group sample sizes, and hence the contrast variances, differ.

Calculating Correlations Carefully

A useful check on the arithmetic is to calculate a correlation in either of two equivalent ways: divide the covariance by the product of standard deviations, or use the formula in terms of sample sizes. The second form makes the dependence on allocation especially clear. For instance, in the first example,

\rho_{12} = \frac{\sqrt{24(36)}}{\sqrt{(40+24)(40+36)}} = \frac{29.394}{69.742} \approx 0.4215.

This agrees with the matrix calculation. A correlation must lie between \(-1\) and \(1\); here all three are positive and less than one. Under the stated shared-control model, positive off-diagonal entries are expected. The correlations also need not be equal even though all three contrasts use the same control group.

Worked Example: A Different Unequal Allocation

Consider a second hypothetical design with \(n_0=30\), \(n_1=15\), \(n_2=30\), and \(n_3=45\), and common variance \(\sigma^2=64\) outcome-units squared. The contrast variances are \(64(1/15+1/30)=6.400\), \(64(1/30+1/30)=4.267\), and \(64(1/45+1/30)=3.556\). The shared covariance is \(64/30=2.133\), so

\boldsymbol{\Sigma}_D= \begin{pmatrix} 6.400&2.133&2.133\\ 2.133&4.267&2.133\\ 2.133&2.133&3.556 \end{pmatrix}.

The resulting correlations are \(\rho_{12}=2.133/\sqrt{6.400(4.267)}=0.4082\), \(\rho_{13}=2.133/\sqrt{6.400(3.556)}=0.4472\), and \(\rho_{23}=2.133/\sqrt{4.267(3.556)}=0.5477\). These calculations show why one should not assign a single common correlation to every pair merely because the control group is shared: the denominators differ from pair to pair.

Using an Estimated Common Variance

In practice, \(\sigma^2\) may not be known. Under the common-variance model, a pooled within-group variance estimate can be substituted into the covariance formula. For group sample variances \(s_g^2\), with \(g=0,1,2,3\), the pooled estimate is

s_p^2=\frac{\sum_{g=0}^{3}(n_g-1)s_g^2}{\sum_{g=0}^{3}(n_g-1)}.

Replacing \(\sigma^2\) by \(s_p^2\) gives an estimated covariance matrix. In the correlation formula, the common variance factor cancels, so the correlations depend on the sample sizes under this model, not on the value plugged in for the common variance. This cancellation does not remove the need for the common-variance assumption.

Worked Example: Plugging in a Pooled Variance Estimate

Suppose a hypothetical study has \(n_0=20\), \(n_1=10\), \(n_2=20\), and \(n_3=40\). Assume the four observed within-group sample variances are each \(25\) outcome-units squared. The pooled estimate is \(s_p^2=[(19+9+19+39)25]/(19+9+19+39)=25\). Using \(25\) in place of \(\sigma^2\), the estimated contrast variances are \(25(1/10+1/20)=3.750\), \(25(1/20+1/20)=2.500\), and \(25(1/40+1/20)=1.875\). The estimated covariance between every distinct pair is \(25/20=1.250\). Thus,

\widehat{\boldsymbol{\Sigma}}_D= \begin{pmatrix} 3.750&1.250&1.250\\ 1.250&2.500&1.250\\ 1.250&1.250&1.875 \end{pmatrix}.

The estimated correlations are \(1.250/\sqrt{3.750(2.500)}=0.4082\), \(1.250/\sqrt{3.750(1.875)}=0.4714\), and \(1.250/\sqrt{2.500(1.875)}=0.5774\). These are plug-in estimates of covariance and correlation under the stated model. The covariance estimates carry squared outcome units; the correlations do not.

Checks, Comparisons, and Limitations

A quick structural check is to confirm that the covariance matrix is symmetric, has the calculated contrast variances on its diagonal, and has \(\sigma^2/n_0\) in every off-diagonal position. Under positive sample sizes and \(\sigma^2>0\), the matrix is positive definite: for any nonzero vector \(a=(a_1,a_2,a_3)\),

a^\mathsf{T}\boldsymbol{\Sigma}_D a = \sigma^2\left(\frac{a_1^2}{n_1}+\frac{a_2^2}{n_2}+\frac{a_3^2}{n_3} +\frac{(a_1+a_2+a_3)^2}{n_0}\right)>0.

Each term is nonnegative, and at least one of the first three terms is positive for a nonzero \(a\). This also explains why a covariance matrix assembled from these formulas is valid when the assumptions hold.

With separate independent control groups for the treatment comparisons, there would be no shared control mean and the off-diagonal covariances would be zero under independent groups. With a shared control, the off-diagonal terms are nonzero, and analyses that assume independent contrasts do not reflect this model. General contrast covariance formulas provide a broader way to handle other contrast definitions; here the direct calculation is simpler because each contrast has one treatment mean and the same control mean.

The calculation depends on independent group means and a common within-group variance. If variances differ by group, the diagonal and shared-control covariance need to be reconsidered using the group-specific variances; the common-\(\sigma^2\) formula should not be applied automatically. Likewise, dependence between observations or overlap between groups changes the covariance calculation. Normality is required for exact joint normality, but not for the covariance formulas themselves when the variances and independence assumptions hold.

Common Mistakes

Setting the off-diagonal covariance equal to zero. That ignores the control mean appearing in both contrasts. Under this model, the off-diagonal covariance is \(\sigma^2/n_0\).
Using the same variance for all contrasts. Unequal treatment sample sizes generally give unequal diagonal entries, because each includes \(\sigma^2/n_i\).
Calling covariance and correlation interchangeable. Covariance is measured in squared outcome units and depends on the scale; correlation is standardized and dimensionless.
Assuming unequal sample sizes mean unequal variances. Unequal sample sizes affect the variances of sample means even when the underlying group outcome variance is common.
Using the formula without checking the design assumptions. A shared control alone is not sufficient if observations are dependent or group variances differ from the assumed common value.

Key takeaway: For three independent treatment groups sharing one independent control group with common outcome variance, each contrast variance is \(\sigma^2(1/n_i+1/n_0)\), while every distinct-pair covariance is \(\sigma^2/n_0\). Unequal sample sizes make the pairwise correlations depend on which two treatment arms are being compared.

Check Your Understanding

Try each question, then compare your work with the solutions below.

Under the common-variance shared-control model, what is \(\operatorname{Cov}(D_1,D_3)\) when \(n_0=25\) and \(\sigma^2=80\)?
For \(n_0=20\), \(n_1=10\), and \(n_2=40\), write the formula for \(\rho_{12}\) and calculate it to three significant figures.
Why can three shared-control contrasts have different pairwise correlations even though their off-diagonal covariances are equal?
Name two assumptions needed for the covariance matrix formula, and state one way the calculation could change if an assumption fails.

Solutions. (1) The covariance is \(\sigma^2/n_0=80/25=3.20\) outcome-units squared. (2) The correlation is \(\sqrt{10(40)}/\sqrt{(20+10)(20+40)}=\sqrt{400}/\sqrt{1800}=20/42.43=0.471\). (3) Each correlation divides the common covariance by the product of the two contrasts’ standard deviations, and those standard deviations vary with the treatment sample sizes. (4) Required assumptions include independence of observations and groups, and a common group variance. If group variances differ, the shared-control contribution and each contrast variance must be computed using the relevant group-specific variances rather than a single \(\sigma^2\).