Deriving The Variance-Based Eta Shrinkage Formula

Why Compare Two Variances?

An empirical Bayes estimate (EBE) for an individual’s random effect is informed by both that individual’s observations and the population model. The resulting estimates may vary less across individuals than the underlying random effects are assumed to vary in the population. Eta shrinkage summarizes that reduction in spread for a particular random-effect component.

The earlier material on defining eta shrinkage from empirical Bayes estimates introduced the idea of comparing EBE dispersion with population random-effect dispersion. Here, the comparison is derived explicitly from the variances. The key point is that the common formula measures the fraction of standard deviation not retained by the EBEs, even though the calculation may begin with variances.

Assumptions and Quantities in the Formula

In a standard population model, the random effect for individual \(i\) is written as \(\eta_i\), with population mean zero and variance \(\omega^2\). For the calculation, the model provides an estimate \(\hat\omega^2>0\). This is a variance, so its square root, \(\sqrt{\hat\omega^2}\), is the estimated population standard deviation of that random-effect component.

The EBEs, written \(\hat\eta_i\), are on the same scale as \(\eta_i\). Their sample variance is calculated across the same set of individuals used for the comparison:

This definition centers the variance on the sample mean of the EBEs. A finite set of EBEs need not have a mean exactly equal to zero, even when the population random effect is modeled with mean zero. Centering makes the calculation a measure of their observed spread rather than a mixture of spread and offset. Software implementations can differ in their variance convention, so a reported value should be interpreted using the convention actually used.

The formula is meaningful when \(\hat\omega^2\) is positive, and it requires the two variances to refer to the same random-effect component, parameterization, and scale. For example, if the random effect is defined on a log scale, both variances must be on that log scale. If a random-effect covariance matrix is used, the comparison for one component uses its marginal variance, not a variance from a different component.

The arithmetic itself does not require the EBEs to be normally distributed. The usual population-model interpretation does rely on treating \(\hat\omega^2\) as the model’s estimate of the population variability for the specified random effect. The formula is a descriptive comparison, not a test of a null hypothesis and not a direct estimate of each individual’s true random effect.

Deriving the Standard-Deviation Formula

Start with the relative EBE spread: the standard deviation of the EBEs divided by the estimated population standard deviation. This ratio describes the fraction of population-scale variability represented by the EBE spread. Eta shrinkage is defined as the amount by which that relative spread falls short of one.

Subtracting this retained fraction from one gives the common formula:

The square root is essential: the definition compares standard deviations, not variances. If EBE variance is one quarter of the estimated population variance, then EBE standard deviation is one half of the population standard deviation, and the formula gives \(1-0.5=0.5\), or 50% shrinkage. Simply subtracting the variance ratio from one would give 75%, which is a different quantity.

Worked Calculations

Checking the Calculation and Interpreting Its Limits

A useful arithmetic check is to compare the variance ratio and the retained standard-deviation fraction. If \(r=s^2_{\hat\eta}/\hat\omega^2\), then the retained fraction is \(\sqrt{r}\), and shrinkage is \(1-\sqrt{r}\). For \(0\leq r\leq1\), the estimate lies between zero and one. At \(r=1\), it is zero; at \(r=0\), it is one. If \(r>1\), it is negative.

The metric is a compact summary, but it is not a substitute for checking how it was calculated. Important checks include:

• Match the component and scale. Do not compare one parameter’s EBE spread with another parameter’s random-effect variance, or mix log-scale and natural-scale quantities.
• Verify the variance convention. Confirm whether the EBE variance is centered on the sample mean and whether the denominator is \(n-1\) or another convention. Use the convention stated for the analysis.
• Check the number and identity of individuals. The EBE variance should be calculated across the intended set of individuals, without accidentally mixing estimates from different analyses or components.
• Investigate unusual ratios. A negative value can result from an EBE spread larger than the estimated population spread. A zero estimated population variance makes this ratio undefined; do not divide by zero or present a finite shrinkage value from this formula.

The comparison is sensitive to both quantities in its ratio. It does not isolate why the EBE spread is smaller or larger, and it does not quantify uncertainty in the shrinkage estimate. With few individuals, the sample variance itself may be unstable. In addition, a single summary can conceal the distribution of the EBEs. These limitations matter when comparing values across components or analyses: a difference in the reported percentage is not, by itself, proof of a difference in individual information or model quality.

Common Mistake: Subtracting the Variance Ratio

A frequent error is to calculate \(1-s^2_{\hat\eta}/\hat\omega^2\) and label it the common eta-shrinkage formula. That expression compares variances. The common formula compares standard deviations, so the square root must be applied to the variance ratio before subtracting from one.

For example, if the EBE variance is \(0.20\) and the estimated population variance is \(0.50\), then the variance ratio is \(0.40\). The common estimate is \(1-\sqrt{0.40}=0.3675\), or 36.75%, not \(1-0.40=0.60\), or 60%. Always identify whether the intended measure is defined on the variance scale or standard-deviation scale before interpreting a percentage.