Sampling Error in Component Estimates

Why Component Estimates Need an Error Bar

A variance-component estimate is itself calculated from a finite sample. Even when the model is correct, a different set of risks or observations would generally produce different within- and between-risk mean squares—and therefore different estimates of process variance and hypothetical-mean variance. A reported component estimate is not automatically a precise measurement of the underlying model parameter.

This tutorial quantifies that sampling variation in a balanced repeated-observation setting. The formulas below are exact under the stated normal model. They help distinguish a large estimated component from a precisely estimated one; the two are not the same.

Model and notation: Suppose there are $N$ independent risks, each observed for $r$ periods. For risk $i$ in period $j$, let $$ Y_{ij}=\Theta_i+\epsilon_{ij}, $$ where the risk-specific means are independent with $\Theta_i\sim N(\mu,\tau^2)$, and the errors are independent with $\epsilon_{ij}\sim N(0,\sigma^2)$. Assume the errors are independent of all the $\Theta_i$. The process variance is $\sigma^2$, and the variance of hypothetical means (VHM) is $\tau^2$. All risks have the same number of observations, and all periods have the same variance.

For this model, an observation varies both because risks have different underlying means and because a particular risk’s observations fluctuate around its own mean. These are distinct sources of variation. To estimate them, define the risk average $\bar Y_i$ and the overall average $\bar Y$, then calculate

MSW=\frac{\displaystyle\sum_{i=1}^{N}\sum_{j=1}^{r}(Y_{ij}-\bar Y_i)^2}{N(r-1)}, \qquad MSB=\frac{\displaystyle r\sum_{i=1}^{N}(\bar Y_i-\bar Y)^2}{N-1}.

Here $MSW$ is the within-risk mean square, with $N(r-1)$ degrees of freedom. The between-risk mean square $MSB$ has $N-1$ degrees of freedom; its numerator includes the multiplier $r$.

Estimates and Their Sampling Errors

Under the model, the expected within-risk mean square is the process variance, while the expected between-risk mean square includes both components:

E[MSW]=\sigma^2, \qquad E[MSB]=\sigma^2+r\tau^2.

Equating these expected values to the observed mean squares gives the method-of-moments estimates:

\widehat{\sigma^2}=MSW, \qquad \widehat{\tau^2}=\frac{MSB-MSW}{r}.

The estimates are random because $MSW$ and $MSB$ are random. Under the normal, balanced model, their sampling distributions give the following exact variance formulas:

\operatorname{Var}(MSW)=\frac{2\sigma^4}{N(r-1)}, \qquad \operatorname{Var}(MSB)=\frac{2(\sigma^2+r\tau^2)^2}{N-1}, \qquad \operatorname{Cov}(MSW,MSB)=0.

Consequently, the sampling variances of the component estimates are

\operatorname{Var}(\widehat{\sigma^2}) =\frac{2\sigma^4}{N(r-1)}

\operatorname{Var}(\widehat{\tau^2}) =\frac{2(\sigma^2+r\tau^2)^2}{r^2(N-1)} +\frac{2\sigma^4}{r^2N(r-1)}.

In practice, the unknown parameters in these expressions are replaced by estimates. Since $\widehat{\sigma^2}=MSW$ and $r\widehat{\tau^2}+\widehat{\sigma^2}=MSB$, the resulting estimated standard errors are

\widehat{SE}(\widehat{\sigma^2}) =\sqrt{\frac{2(MSW)^2}{N(r-1)}}

\widehat{SE}(\widehat{\tau^2}) = \sqrt{ \frac{2(MSB)^2}{r^2(N-1)} + \frac{2(MSW)^2}{r^2N(r-1)} }.

Why the Formulas Hold

The reason for the sampling-error formulas is the normal model’s separation of within-risk contrasts from variation among risk averages. Within each risk, subtracting $\bar Y_i$ removes its shared mean $\Theta_i$. The remaining $N(r-1)$ independent normal contrasts have variance $\sigma^2$, so their squared sum divided by $\sigma^2$ has a chi-square distribution with $N(r-1)$ degrees of freedom. Thus

\frac{N(r-1)MSW}{\sigma^2}\sim\chi^2_{N(r-1)}.

A risk average has variance $\tau^2+\sigma^2/r$: its shared risk mean contributes $\tau^2$, and averaging $r$ independent errors contributes $\sigma^2/r$. The $N-1$ orthogonal contrasts among the normal risk averages therefore give

\frac{(N-1)MSB}{\sigma^2+r\tau^2}\sim\chi^2_{N-1}.

For a chi-square random variable with $k$ degrees of freedom, the variance is $2k$. Applying that result to each scaled mean square yields their variances above. The within-risk contrasts and contrasts among risk averages are independent in this balanced normal model, which gives zero covariance between $MSW$ and $MSB$. Finally, subtracting $MSW$ in the VHM estimate explains why its variance contains a second term: uncertainty in the process-variance estimate also contributes to uncertainty in the VHM estimate.

Worked Examples

Worked Example: Estimating Both Components and Their Standard Errors

In a hypothetical study, $N=6$ risks are each observed for $r=3$ periods. The data give $MSW=16$ and $MSB=76$, in squared rating units. Assume the balanced normal model above. Estimate both variance components and their plug-in standard errors.

Method. Use the method-of-moments estimates, then substitute the observed mean squares into the estimated standard-error formulas. The model has no interest or mortality assumption: the relevant distributional assumption is the independent normal random-effects model stated above.

Step 1: Estimate process variance. $$\widehat{\sigma^2}=MSW=16.$$

Step 2: Estimate VHM. The between-risk mean square includes $r$ times the VHM, so $$\widehat{\tau^2}=\frac{76-16}{3}=\frac{60}{3}=20.$$

Step 3: Estimate the process-variance standard error. There are $N(r-1)=6(2)=12$ within-risk degrees of freedom: $$\widehat{SE}(\widehat{\sigma^2})=\sqrt{\frac{2(16)^2}{12}} =\sqrt{\frac{512}{12}} =\sqrt{42.6667}\approx 6.53.$$

Step 4: Estimate the VHM standard error. The between-risk term is $$\frac{2(76)^2}{3^2(6-1)} =\frac{11{,}552}{45} =256.7111,$$ and the within-risk term is $$\frac{2(16)^2}{3^2(6)(3-1)} =\frac{512}{108} =4.7407.$$ Therefore, $$\widehat{SE}(\widehat{\tau^2})=\sqrt{256.7111+4.7407} =\sqrt{261.4518}\approx 16.17.$$

Check and interpretation. The estimated mean squares are consistent with the fitted components: $16+3(20)=76$. The VHM estimate of 20 has a plug-in standard error of about 16.17, so its sampling uncertainty is substantial relative to its estimated value. The standard errors are in squared rating units, the same units as the component estimates. The common mistake this calculation avoids is treating $MSB-MSW$ itself as the VHM estimate; the difference must be divided by $r$.

Worked Example: Separating the Effects of More Risks and More Periods

Consider two hypothetical balanced studies with the same observed $MSW=9$, $MSB=21$, and $r=2$. Study A has $N=5$ risks; Study B has $N=20$ risks. Under the same normal model, compare the estimated components and their standard errors.

Step 1: Estimate the components. In either study, $$\widehat{\sigma^2}=9,\qquad \widehat{\tau^2}=\frac{21-9}{2}=6.$$ The estimated mean-square relation checks: $9+2(6)=21$.

Step 2: Calculate standard errors for Study A. Its within-risk degrees of freedom are $5(2-1)=5$: $$\widehat{SE}(\widehat{\sigma^2})= \sqrt{\frac{2(9)^2}{5}} =\sqrt{32.4}\approx5.69.$$ For the VHM estimate, the between-risk term is $$\frac{2(21)^2}{2^2(5-1)}=\frac{882}{16}=55.125,$$ and the within-risk term is $$\frac{2(9)^2}{2^2(5)(1)}=\frac{162}{20}=8.1.$$ Thus $\widehat{SE}(\widehat{\tau^2})=\sqrt{55.125+8.1} =\sqrt{63.225}\approx7.95.$

Step 3: Calculate standard errors for Study B. Its within-risk degrees of freedom are $20(2-1)=20$: $$\widehat{SE}(\widehat{\sigma^2})= \sqrt{\frac{162}{20}} =\sqrt{8.1}\approx2.85.$$ The between-risk term for VHM is $$\frac{882}{2^2(20-1)}=\frac{882}{76}\approx11.6053,$$ and the within-risk term is $$\frac{162}{2^2(20)(1)}=\frac{162}{80}=2.025.$$ Therefore, $\widehat{SE}(\widehat{\tau^2})= \sqrt{11.6053+2.025}=\sqrt{13.6303}\approx3.69.$

Check and interpretation. Both studies yield the same point estimates, but the study with more independent risks has smaller estimated standard errors for both components. The standard errors were rounded to two decimal places. This comparison isolates the effect of $N$ for these stated summaries; it does not mean that observed mean squares would necessarily stay the same in new data. The trap is to infer precision from the point estimates alone: identical component estimates can have quite different sampling uncertainty.

Worked Example: Accounting for Dependence Between Component Estimates

For a hypothetical balanced study, let $N=6$, $r=3$, $MSW=16$, and $MSB=76$. Find the estimated covariance between the process-variance and VHM estimates. Then calculate the standard error of $3\widehat{\tau^2}+\widehat{\sigma^2}$ and compare it with the result from incorrectly assuming the two component estimates are independent.

Step 1: Identify the shared term. The estimates are $MSW$ and $(MSB-MSW)/r$. Even though $MSW$ and $MSB$ are independent under the model, both component estimates use $MSW$. Therefore, $$ \operatorname{Cov}(\widehat{\sigma^2},\widehat{\tau^2}) =\operatorname{Cov}\left(MSW,\frac{MSB-MSW}{r}\right) =-\frac{\operatorname{Var}(MSW)}{r}. $$

Step 2: Estimate the covariance. Here $$\widehat{\operatorname{Var}}(MSW)=\frac{2(16)^2}{6(3-1)} =\frac{512}{12}=42.6667.$$ Hence $$\widehat{\operatorname{Cov}}(\widehat{\sigma^2},\widehat{\tau^2}) =-\frac{42.6667}{3}\approx-14.22.$$

Step 3: Use the covariance in the linear combination. Since $3\widehat{\tau^2}+\widehat{\sigma^2}=MSB$, its estimated variance is $$\widehat{\operatorname{Var}}(MSB)=\frac{2(76)^2}{6-1} =\frac{11{,}552}{5}=2{,}310.4.$$ Its standard error is $\sqrt{2{,}310.4}\approx48.07$.

Step 4: Check the independence mistake. From the first worked example, the estimated component variances are approximately 42.6667 and 261.4518. If covariance were incorrectly set to zero, the variance of the linear combination would be $$42.6667+3^2(261.4518)=2{,}395.733.$$ The correct covariance contribution is $2(3)(-14.2222)=-85.3332$, giving $$2{,}395.733-85.333=2{,}310.4,$$ as required by the identity with $MSB$. The independence assumption overstates uncertainty here because the shared within-risk mean square induces negative covariance. The check is the exact algebraic identity $3\widehat{\tau^2}+\widehat{\sigma^2}=MSB$.

Interpretation, Limits, and Common Traps

A standard error describes the sampling variation of an estimate across repeated samples under a model; it is not the standard deviation of individual observations. In particular, the process variance estimate and the VHM estimate have different sampling-error formulas and can have very different precision. The number of independent risks $N$ is especially important for learning about variation among hypothetical means, while additional observations per risk increase the information about within-risk process variation.

The formulas are exact only for independent risks, equal numbers of observations per risk, equal period variances, and the specified normal model. With unequal numbers of observations or unequal variances, these balanced-model mean squares do not in general have the stated sampling distributions. With materially nonnormal data, the chi-square results are not exact either. In such settings, use a method appropriate to the design and distribution rather than applying these standard errors mechanically.

Forgetting the divisor $r$. The VHM estimate is $(MSB-MSW)/r$, not $MSB-MSW$. The factor $r$ appears because $MSB$ estimates $\sigma^2+r\tau^2$.
Reporting a component without its uncertainty. A method-of-moments estimate is based on finite data. Its size alone does not say how stable it is.
Assuming component estimates are independent. Both estimates use $MSW$, so their covariance is negative under this model. This matters for the uncertainty of combinations of the estimates.
Using exact normal-model formulas outside their conditions. Unbalanced or nonnormal data require more care; the formulas here should not be treated as universal.

Key takeaway: Under a balanced normal random-effects model, sampling error can be quantified from the mean squares and their degrees of freedom. Report component estimates with their standard errors, account for dependence when combining them, and state the design and distributional assumptions that justify the calculation.

Check Your Understanding

Use the balanced normal model and plug-in standard-error formulas unless a question asks otherwise. Round standard errors to two decimal places.

A hypothetical study has $N=10$, $r=3$, $MSW=12$, and $MSB=42$. Find both component estimates and their plug-in standard errors. (Answer: $\widehat{\sigma^2}=12$, $\widehat{\tau^2}=10$, $\widehat{SE}(\widehat{\sigma^2})=3.79$, $\widehat{SE}(\widehat{\tau^2})=6.72$.)
A study has $N=12$, $r=2$, and $MSW=20$. Find the plug-in standard error of the process-variance estimate. (Answer: $8.16$.)
A study has $N=8$, $r=4$, $MSW=9$, and $MSB=45$. Estimate VHM and its plug-in standard error. (Answer: $\widehat{\tau^2}=9$, $\widehat{SE}(\widehat{\tau^2})=6.05$.)
For $N=9$, $r=3$, and $MSW=10$, calculate the estimated covariance between the two component estimates. (Answer: $-3.70$.)
With $MSW=16$ and $N=6$, compare the process-variance standard error for $r=2$ and $r=4$. (Answer: $9.24$ for $r=2$ and $5.33$ for $r=4$.)

What You'll Learn