Balanced Method-Of-Moments Estimates

Why Estimate Two Variance Components?

In a one-way random-effects study, observations from the same group may resemble one another because they share a group-level deviation. The model separates this between-group variation from the residual variation among observations within a group. Estimating both components helps describe how much variability is associated with each source.

This tutorial focuses on a balanced design: every sampled group contributes the same number of observations. In this setting, the expected mean squares give two equations for the two variance components. The method of moments estimates each component by replacing its expected mean square with the observed mean square and solving those equations.

The Balanced One-Way Random-Effects Model

Definition: Suppose $a$ groups are sampled, with $n$ observations in each group. The balanced one-way random-effects model is $$ Y_{ij}=\mu+U_i+\varepsilon_{ij},\qquad i=1,\ldots,a,\quad j=1,\ldots,n. $$ Here $\mu$ is the overall mean, $U_i$ is the random deviation for group $i$, and $\varepsilon_{ij}$ is the residual for observation $j$ in group $i$. The group deviations are independent with mean $0$ and variance $\sigma_A^2$; the residuals are independent with mean $0$ and variance $\sigma^2$; and the two sets of random terms are independent. The between-group variance component is $\sigma_A^2$, and the residual variance component is $\sigma^2$.

The model also commonly assumes that the group deviations and residuals are normally distributed. Normality is needed for the usual exact small-sample ANOVA $F$ procedures, but the method-of-moments equations below follow from the stated means, variances, and independence. The design must have independent groups, the same $n$ observations per group, and the same residual variance across groups. As in other random-effects models, the groups are treated as a sample from a broader population of possible groups.

Let $\bar{Y}_{i\cdot}$ be the mean of the $n$ observations in group $i$, and let $\bar{Y}_{\cdot\cdot}$ be the mean of all $an$ observations. The between-group and residual sums of squares are

SS_A=n\sum_{i=1}^{a}(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2,\qquad SS_E=\sum_{i=1}^{a}\sum_{j=1}^{n}(Y_{ij}-\bar{Y}_{i\cdot})^2.

The corresponding degrees of freedom are $a-1$ for groups and $a(n-1)$ for residuals. Dividing each sum of squares by its degrees of freedom gives the mean squares:

MS_A=\frac{SS_A}{a-1},\qquad MS_E=\frac{SS_E}{a(n-1)}.

From Expected Mean Squares to Estimates

For the balanced random-effects model, the expected residual mean square is the residual variance, while the expected between-group mean square contains both variance components:

E(MS_E)=\sigma^2,\qquad E(MS_A)=\sigma^2+n\sigma_A^2.

The multiplier $n$ occurs because each group-level deviation is shared by all $n$ observations in its group. Consequently, more observations per group make between-group variation contribute more strongly to the between-group mean square.

Definition: The method-of-moments estimates are obtained by equating each observed mean square to its corresponding expected mean square and solving the resulting equations for the unknown population parameters. In the balanced one-way random-effects model, $$ \widehat{\sigma^2}_{\mathrm{MOM}}=MS_E,\qquad \widehat{\sigma_A^2}_{\mathrm{MOM}}=\frac{MS_A-MS_E}{n}. $$ Both estimates are expressed in squared units of the measured outcome.

The residual estimate is $MS_E$ because its expected value is $\sigma^2$. To obtain the between-group estimate, set $MS_A=\sigma^2+n\sigma_A^2$ and $MS_E=\sigma^2$. Subtracting the second equation from the first removes the residual component, leaving $MS_A-MS_E=n\sigma_A^2$. Dividing by $n$ gives the stated estimate. This subtraction is essential: $MS_A$ includes residual variation as well as between-group variation.

Worked Example: Three Groups With Two Observations Each

Suppose a hypothetical laboratory exercise measures an outcome in units on three randomly sampled groups, with two independent observations per group. The observed values are Group A: 8 and 10; Group B: 10 and 12; Group C: 12 and 14. Assume the balanced random-effects model: the groups are independent, the residuals are independent with a common variance, and the random group deviations are independent of the residuals. The measurements alone cannot verify these design and model assumptions.

The group means are 9, 11, and 13, and the grand mean is 11. With $a=3$ and $n=2$, the between-group sum of squares is

SS_A=2\left[(9-11)^2+(11-11)^2+(13-11)^2\right] =2(4+0+4)=16.

Within each group, the deviations from its own mean are $-1$ and $1$, so each group contributes $1+1=2$ to $SS_E$. Therefore,

SS_E=2+2+2=6,\qquad MS_A=\frac{16}{3-1}=8,\qquad MS_E=\frac{6}{3(2-1)}=2.

The method-of-moments estimates are

\widehat{\sigma^2}_{\mathrm{MOM}}=2,\qquad \widehat{\sigma_A^2}_{\mathrm{MOM}}=\frac{8-2}{2}=3.

Thus the estimated residual variance is 2 squared outcome-units, and the estimated between-group variance is 3 squared outcome-units. These are estimates of population variance components, not the observed range of the group means.

A Second Balanced Calculation

A useful check on the formulas is to calculate them when there are more groups and more observations per group. The same equations apply as long as every group has the same number of observations.

Worked Example: Four Groups With Three Observations Each

Suppose a hypothetical instrument is used to record a measurement in units for three observations in each of four randomly sampled groups. The data are Group A: 19, 20, 21; Group B: 21, 22, 23; Group C: 23, 24, 25; and Group D: 25, 26, 27. Assume independent sampled groups, independent residuals with a common variance, and independent random group deviations and residuals. Each group has exactly three observations, so the design is balanced.

The group means are 20, 22, 24, and 26, and the grand mean is 23. The between-group sum of squares is

SS_A=3\left[(20-23)^2+(22-23)^2+(24-23)^2+(26-23)^2\right] =3(9+1+1+9)=60.

In each group, the deviations from that group’s mean are $-1,0,1$, giving a within-group sum of squared deviations of $1+0+1=2$. Across four groups, $SS_E=4(2)=8$. Thus,

MS_A=\frac{60}{4-1}=20,\qquad MS_E=\frac{8}{4(3-1)}=\frac{8}{8}=1.

Substitute these mean squares and $n=3$ into the component formulas:

\widehat{\sigma^2}_{\mathrm{MOM}}=1,\qquad \widehat{\sigma_A^2}_{\mathrm{MOM}}=\frac{20-1}{3} =\frac{19}{3}\approx 6.333.

The estimated residual variance is 1 squared measurement-unit, and the estimated between-group variance is about 6.333 squared measurement-units. The division by 3 reflects the fact that three observations per group contribute to the expected between-group mean square.

Reading Estimates From a Balanced ANOVA Table

When a balanced ANOVA table reports mean squares rather than the raw observations, no further sum-of-squares calculation is needed. The table must still identify the group and residual mean squares and the common number of observations per group.

Worked Example: Estimating Components From Mean Squares

A hypothetical balanced study measures an outcome in millimetres in three randomly sampled groups, with four observations in each group. Its ANOVA table reports $MS_A=17.333$ and $MS_E=0.667$. Assume the groups and residuals satisfy the independent random-effects model with a common residual variance. Here $n=4$, so the estimates are

\widehat{\sigma^2}_{\mathrm{MOM}}=MS_E=0.667,

and

\widehat{\sigma_A^2}_{\mathrm{MOM}} =\frac{MS_A-MS_E}{n} =\frac{17.333-0.667}{4} =\frac{16.666}{4} \approx 4.167.

The estimated residual variance is $0.667$ square millimetres, and the estimated between-group variance is approximately $4.167$ square millimetres. The group mean square alone is not the between-group variance estimate: it includes the residual component, and its excess over $MS_E$ must be divided by the replication count.

How to Apply the Method

Confirm the balanced design.
Identify the number of groups $a$ and verify that each has the same number $n$ of observations. The formulas here are for equal group sizes.

Obtain the two mean squares.
Use the between-group and residual rows of the ANOVA table, or calculate their sums of squares and divide by the appropriate degrees of freedom.

Estimate the residual component.
Set $\widehat{\sigma^2}_{\mathrm{MOM}}=MS_E$. Retain squared outcome-units in the interpretation.

Estimate the between-group component.
Subtract $MS_E$ from $MS_A$, then divide the difference by the common replication count $n$.

Report in context.
Name the source of each variance component and give its units. Do not describe a variance estimate as a standard deviation.

Common Mistakes and Why the Estimates Matter

The two estimates describe different sources of variability. The residual component concerns variation among observations within a group after accounting for the group deviation. The between-group component concerns variation in the deviations of sampled groups from the overall mean. Reporting both gives a more informative description than reporting only the total spread of all observations.

Using $MS_A$ as the between-group variance estimate. This ignores the residual contribution in $E(MS_A)=\sigma^2+n\sigma_A^2$. Subtract $MS_E$ before dividing by $n$.
Dividing by the number of groups instead of the replication count. The coefficient of $\sigma_A^2$ in the expected group mean square is $n$, the number of observations per group. Therefore, the difference $MS_A-MS_E$ is divided by $n$.
Confusing variance with standard deviation. Both method-of-moments estimates are variances and have squared units. A standard deviation would be on the original measurement scale and is not what these formulas directly estimate.
Assuming the design is balanced because the total sample size is divisible by the number of groups. Balance requires the same number of observations in every group, not merely an evenly divisible total. The unequal-group-size case requires different calculations.

The estimates are based on observed mean squares, so they vary from sample to sample. In some samples, $MS_A$ can be smaller than $MS_E$, making the raw method-of-moments estimate of the between-group variance negative. A variance component in the model cannot itself be negative; such an estimate signals sampling variability and requires careful handling rather than an automatic, unexplained alteration. The formula shown here is the direct method-of-moments solution.

Key takeaway: For a balanced one-way random-effects design with $n$ observations per group, estimate residual variance by $MS_E$ and between-group variance by $(MS_A-MS_E)/n$. The subtraction removes the residual contribution to the group mean square, while the division accounts for the number of observations sharing each group effect.

Check Your Understanding

Use the balanced random-effects model and its method-of-moments formulas to answer these questions.

A balanced study has five groups with four observations per group. If $MS_A=13$ and $MS_E=5$, calculate both variance-component estimates and state their units relative to the outcome.
Explain why the estimate of $\sigma_A^2$ uses $MS_A-MS_E$ rather than $MS_A$ alone.
If each group contains six observations, what quantity divides $MS_A-MS_E$ when estimating the between-group component?
A report labels $MS_E$ as the estimated residual standard deviation. What is incorrect about this label?
Why is having the same total sample size in two studies not enough to establish that both designs are balanced?