Cohen's f for One-Way Effects

Why Standardize the Separation Among Group Means?

An omnibus ANOVA asks whether population group means are all equal. An effect size answers a different question: how far apart are the group means, relative to the variation of individual outcomes within groups? The answer depends on both the pattern of means and the scale of within-group variability. The same differences between means can be substantial when outcomes vary little within groups and modest when outcomes vary greatly.

Cohen’s \(f\) is a dimensionless measure of that standardized separation for a one-way effect. It uses the weighted spread of the population group means around their grand mean, divided by the common within-group standard deviation. The weights matter: they describe the group proportions for the population or design being considered.

Definition and Model

Let there be \(k\) groups. Let \(\mu_i\) be the population mean in group \(i\), and let \(w_i\) be that group’s weight, with \(w_i \geq 0\) and \(\sum_{i=1}^{k}w_i=1\). Define the weighted grand mean as \(\mu=\sum_{i=1}^{k}w_i\mu_i\). Suppose the outcome has the same within-group population standard deviation \(\sigma>0\) in every group.

The numerator is the weighted standard deviation of the group means, measured in the outcome’s units. The denominator is the within-group standard deviation, in those same units. Dividing makes \(f\) unitless. If all group means are equal, the numerator and \(f\) are zero. As the means spread farther apart relative to \(\sigma\), \(f\) grows.

A standard one-way model for using this definition is \(Y_{ij}=\mu_i+\varepsilon_{ij}\), where \(Y_{ij}\) is the outcome for observation \(j\) in group \(i\). The usual model assumes independent errors with mean zero, a common variance \(\sigma^2\), and normally distributed errors for exact classical ANOVA inference. Thus observations are independent, outcomes within each group are normally distributed under the model, and the within-group variances are equal. Cohen’s \(f\) is defined from population means and a common population standard deviation; these assumptions describe the standard model in which the effect is used.

Weights and Unequal Group Proportions

For equally represented groups, equal weights are appropriate. When groups have unequal proportions in the target population or design, use those proportions rather than treating the groups as equally weighted. The resulting grand mean and mean dispersion describe the group mixture represented by those weights.

For example, if the target proportions are 0.50, 0.30, and 0.20, the grand mean is \(0.50\mu_1+0.30\mu_2+0.20\mu_3\). It is not generally the unweighted average of the three means. A different choice of weights can produce a different \(f\), even if the means and common standard deviation stay the same.

Connection to Eta-Squared

Population eta-squared, denoted here by \(\eta^2\), is the proportion of the total outcome variance in the weighted group mixture that is attributable to differences among the group means. Under the common-variance model, the total variance separates into the weighted variance of group means plus the common within-group variance. In symbols, if \(G\) denotes group membership, then

The first term is the between-group component, and the second is the within-group component. By definition, population eta-squared is the first term divided by total variance. Let \(B=\sum_{i=1}^{k}w_i(\mu_i-\mu)^2\). Then \(\eta^2=B/(B+\sigma^2)\), while \(f^2=B/\sigma^2\). Substituting and rearranging gives the relationship.

This relationship can also be derived directly: from \(\eta^2=B/(B+\sigma^2)\), it follows that \(1-\eta^2=\sigma^2/(B+\sigma^2)\). Therefore \(\eta^2/(1-\eta^2)=B/\sigma^2=f^2\). Taking the nonnegative square root yields \(f=\sqrt{\eta^2/(1-\eta^2)}\). This is an exact population relationship, not a claim that the two measures use different information about the group-mean separation.

Calculating a Sample-Based Value

In practice, population means and \(\sigma\) are usually unknown. A one-way ANOVA table provides the between-groups sum of squares, \(SS_{\text{between}}\), and the total sum of squares, \(SS_{\text{total}}\). The sample eta-squared is \(\eta^2=SS_{\text{between}}/SS_{\text{total}}\). A descriptive sample-based value of \(f\) can be obtained by applying the population conversion to this sample eta-squared:

Here \(SS_{\text{within}}=SS_{\text{total}}-SS_{\text{between}}\). This is a plug-in effect-size calculation from the observed sample, not an unbiased estimate of population \(f\). Sampling variability can make sample effect sizes differ from the population effect. In particular, a sample eta-squared and the corresponding \(\hat f\) should not be described as known population values.

Interpretation and Common Pitfalls

Cohen’s \(f\) is nonnegative and dimensionless. It summarizes the overall spread of weighted group means relative to within-group variability; it does not show which groups differ, the direction of any difference, or the size of a difference in the outcome’s original units. Two very different patterns of group means can yield the same \(f\). Use the means and suitable comparisons when those details matter.

Cohen’s conventional reference values of \(f=0.10\), \(0.25\), and \(0.40\) are sometimes described as small, medium, and large. They are rough guides, not universal boundaries. Whether an effect matters depends on the outcome, context, and consequences. A value should not be called important solely because it crosses one of these reference points.

A common mistake is to say, “The sample eta-squared is 0.20, so Cohen’s \(f\) is also 0.20.” That confuses a variance proportion with a standardized standard deviation. The correct conversion is \(f=\sqrt{\eta^2/(1-\eta^2)}\); for \(\eta^2=0.20\), it gives \(f=0.500\). Another mistake is to use equal weights automatically when the group proportions are unequal. The weights must match the population or design being summarized.