Estimating Contrast Covariance From a Common Variance

Why Estimate a Shared-Control Covariance Matrix?

When several treatment groups are compared with the same control group, each treatment-control difference contains the control sample mean. Consequently, the estimated differences are generally dependent. Their theoretical covariance matrix depends on the common outcome variance, which is usually unknown. This tutorial shows how to estimate that matrix by estimating the common variance from within-group variation and substituting the estimate into the covariance formulas.

The target here is the covariance matrix of the estimated contrasts, not the covariance of individual patient outcomes and not an uncertainty interval for an estimated correlation. The plug-in calculation is useful when a common-variance model is appropriate: it gives a single matrix that reflects both the estimated variability and the allocation of participants across groups.

The Pooled Variance and the Plug-In Estimate

The common variance is estimated by pooling the within-group sums of squares. The group means do not enter this calculation: they describe between-group differences, whereas the pooled estimate uses variation around each group’s own mean. Write \(N=\sum_{j=0}^K n_j\) for the total sample size.

The denominator \(N-(K+1)\) is the residual degrees of freedom: there are \(K+1\) groups, and each group mean is estimated from its own data. Under the stated normal, independent, common-variance model, \(s_p^2\) is unbiased for \(\sigma^2\). Substituting \(s_p^2\) for \(\sigma^2\) in the covariance formulas produces the estimated contrast covariance matrix.

The full matrix can be written compactly. Let \(C\) be the \(K\)-by-\((K+1)\) matrix whose row \(j\) has coefficient \(1\) for treatment group \(j\), coefficient \(-1\) for control group \(0\), and zero elsewhere. Let \(M=(\bar{Y}_0,\bar{Y}_1,\ldots,\bar{Y}_K)^\mathsf{T}\), and let \(V_n=\operatorname{diag}(1/n_0,\ldots,1/n_K)\). Then \(D=CM\). The estimate is:

This matrix form also applies when the target contrasts are not all treatment versus control. If the rows of a matrix \(A\) define the desired linear contrasts of the group means, the corresponding plug-in estimate is \(s_p^2 A V_n A^\mathsf{T}\). Each row of \(A\) specifies which group means are added or subtracted and with what coefficients.

Why the Substitution Works

Independence across groups implies that the covariance matrix of \(M\) is diagonal: the variance of \(\bar{Y}_j\) is \(\sigma^2/n_j\), and different group means have covariance zero. For any fixed contrast matrix \(A\), the covariance of \(AM\) is therefore obtained by multiplying this diagonal matrix on the left by \(A\) and on the right by \(A^\mathsf{T}\).

Under independent normal observations with a common variance, the within-group residual sum of squares in group \(j\), divided by \(\sigma^2\), has a chi-square distribution with \(n_j-1\) degrees of freedom. These residual sums of squares are independent across groups. Their sum, divided by \(\sigma^2\), therefore has a chi-square distribution with \(\sum_j(n_j-1)=N-(K+1)\) degrees of freedom. Its expectation is that degrees of freedom, so \(E(s_p^2)=\sigma^2\). Consequently, replacing \(\sigma^2\) by \(s_p^2\) gives an unbiased estimate of each covariance entry:

This justification depends on the common-variance model. It establishes unbiasedness of the covariance estimator under that model; it does not mean that every element of the estimated matrix is known without sampling error.

Using Group Summaries and General Contrasts

Often the available report gives each group’s sample size and sample variance rather than the individual outcomes. The weighted pooling formula uses each group variance multiplied by its within-group degrees of freedom, not a simple average of the group variances.

For other contrasts, use their coefficients directly rather than trying to adjust the treatment-control formula by inspection. For example, let \(D_1=\bar{Y}_1-\bar{Y}_0\), \(D_2=\bar{Y}_2-\bar{Y}_0\), and \(D_3=\bar{Y}_3-\bar{Y}_0\). The contrast \(\bar{Y}_1-\bar{Y}_2\) equals \(D_1-D_2\); its control component cancels. A matrix multiplication automatically handles that cancellation and preserves the correct covariance with any other contrast.

Assumptions, Diagnostics, and Alternatives

Before using the plug-in matrix, check that its model matches the design and outcome. Independent observations are needed for the group-mean covariance calculation. The pooled variance also assumes a common within-group variance. Normality is needed for the exact chi-square justification for \(s_p^2\); substantial skewness or influential outliers can make the pooled estimate and normal-model inference unreliable, especially in small groups.

Useful checks include examining residuals within each group, comparing group spreads, and identifying outliers or data errors. Notice that sample variances will not be identical even when population variances are equal. Diagnostics are evidence about the plausibility of the assumption, not a proof that it holds. If group variances are markedly different, a common pooled estimate can give a poor representation of the covariance structure.

These choices are not interchangeable. The shared-control structure determines which group means appear together in each contrast, while the variance model determines how the uncertainty of those means is estimated. Changing the variance model does not remove the shared-control contribution; it changes the appropriate covariance calculation. A simple pooled plug-in matrix is limited when the common-variance assumption is implausible, when observations are dependent in a way the model omits, or when the data do not support stable within-group variance estimates.

Common Pitfalls

• Using the wrong denominator. Dividing the total within-group sum of squares by \(N\) or \(N-1\) ignores the group means estimated from the data. Use \(N-(K+1)\).
• Averaging variances without weights. Group variances have different degrees of freedom when sample sizes differ. Pool \((n_j-1)s_j^2\) and divide by the sum of the degrees of freedom.
• Putting zero in the off-diagonal entries. Treatment-control contrasts share the control mean, so their covariance is generally nonzero under this model.
• Calling the estimate a correlation matrix. The entries have squared outcome units. Correlations require division by the relevant standard deviations.
• Treating pooled variance as assumption-free. The formula can be calculated even when variances differ, but its common-variance interpretation then lacks its model justification.