Planned Weighted Comparisons Across Several Groups

Why Plan a Weighted Comparison?

When a regression model represents several groups, the question of interest is not always whether two particular groups differ. A researcher might instead want to compare one group with the average of several others, or compare the average of two groups with a third. A planned weighted contrast states that comparison directly, using specified weights on the group means.

The weights encode the scientific question. For example, comparing one group with an equally weighted average of three others gives the three comparison groups equal influence, regardless of their sample sizes. That is different from comparing it with a sample-size-weighted pooled average. The choice should reflect the target comparison, not whichever weights make an observed result look strongest.

The zero-sum condition makes the contrast a comparison rather than an overall level. If the same constant \(a\) is added to every group mean, then \( \sum_j c_j(\mu_j+a)=\sum_j c_j\mu_j+a\sum_j c_j=L \), because \(\sum_j c_j=0\). The contrast therefore does not change just because the outcome has a different common reference level.

Choosing Weights for a Group Versus a Weighted Average

To compare group 1 with a weighted average of groups 2 through \(k\), choose nonnegative weights \(w_2,\ldots,w_k\) that sum to one. Set \(c_1=1\) and \(c_j=-w_j\) for the other groups. Then \( L=\mu_1-\sum_{j=2}^{k}w_j\mu_j \). The coefficients sum to zero, and the second term is the specified weighted average.

For an equal-weight average of three comparison groups, each receives weight \(1/3\), so the contrast coefficients are \(1,-1/3,-1/3,-1/3\). For a comparison of group 1 with 25% of group 2 and 75% of group 3, the coefficients are \(1,-0.25,-0.75\). A coefficient’s sign and size matter: reversing every sign reverses the direction of the contrast but leaves its two-sided test unchanged.

The choice of weights also identifies what population summary is being compared. Equal weights answer a question about the average of group means, giving each group equal standing. Weights proportional to group population shares may answer a different question. Neither choice is automatically correct in every setting; the intended comparison determines the weights.

Standard Errors and the Contrast Test

State the model assumptions before using a t test. In a one-way regression model for \(k\) groups, assume observations are independent, the mean is correctly represented by the group-specific means, and errors within groups are normally distributed with a common variance \(\sigma^2\). The usual pooled estimate of that variance is the residual mean square, denoted MSE, and the residual degrees of freedom are \(df=n-k\). Under these assumptions the t test and confidence interval below have their stated exact t distributions. In a regression with covariates, the mean model and error assumptions must instead be appropriate for that fitted regression; adjusted group means can be correlated, so their covariance must be included.

Let \(\hat{\boldsymbol{\mu}}\) be the vector of estimated means and let \(\widehat{\operatorname{Cov}}(\hat{\boldsymbol{\mu}})\) be its estimated covariance matrix. For coefficient vector \(\mathbf{c}\), the estimate is \(\hat{L}=\mathbf{c}^{\mathsf T}\hat{\boldsymbol{\mu}}\), and its estimated variance is the quadratic form below. The covariance terms are essential whenever the estimated means are correlated.

For independent group sample means from the one-way model, the estimated covariance matrix is diagonal, with the variance estimate for group \(j\)'s sample mean equal to \(\mathrm{MSE}/n_j\). Thus the contrast variance simplifies to \(\mathrm{MSE}\sum_j c_j^2/n_j\). This formula uses squared coefficients: signs determine the contrast’s direction, but variance contributions from independent means cannot be negative.

To test \(H_0:L=0\) against a two-sided alternative \(H_a:L\ne 0\), calculate \(t=\hat{L}/SE(\hat{L})\). Under the stated model and null hypothesis, this statistic follows a t distribution with the regression residual degrees of freedom. A confidence interval is \(\hat{L}\pm t^*SE(\hat{L})\), using the appropriate t critical value. The estimate and interval retain the outcome’s units; the t statistic does not.

Worked Comparisons

Planning, Interpretation, and Common Pitfalls

A planned contrast is useful because its weights make the scientific comparison explicit and interpretable. It is not a license to search through many possible contrasts after seeing the data and report only the most favorable one. When several contrasts are examined, the interpretation of their inferential results requires additional care; the single-contrast calculations here do not, by themselves, address that broader issue.

• Do not confuse equal weights with equal sample sizes. An equal-weight average of group means gives each group the same weight even if their sample sizes differ. Use the weights that match the question.
• Do not omit covariance terms. For correlated adjusted means, use the full covariance matrix. A formula based only on separate standard errors assumes zero covariance and may be inappropriate.
• Do not interpret a contrast as a causal effect automatically. A regression contrast describes a difference in modeled means. Causal interpretation requires a suitable study design and assumptions beyond the contrast calculation.
• Do not treat a nonsignificant result as proof of equality. A wide interval can include zero and still include differences that matter. Report the estimate and uncertainty in the outcome’s units.