Worked Eta and Epsilon Shrinkage Calculations

Why Work Through Shrinkage Calculations?

Shrinkage summarizes how much the spread of individual estimates or residuals falls below a model-based variability scale. The calculation is simple, but its meaning depends on which spread and denominator are used. In particular, a sample standard deviation, a population standard deviation, and a variance do not give interchangeable shrinkage values. A transparent calculation therefore states the data, the scale, and the convention before interpreting the result.

This tutorial works through eta shrinkage for empirical Bayes estimates (EBEs) and epsilon shrinkage for conditional residuals. All datasets and model values below are hypothetical. The examples use a homogeneous residual-error model for epsilon shrinkage and a single random-effect component at a time for eta shrinkage.

What Goes Into Each Calculation?

An EBE is an individual estimate of a subject’s random effect, conditional on that subject’s data and the fitted model. For a particular random-effect component, calculate the spread across subjects and compare it with the fitted population SD for that same component. For example, EBEs for a log-clearance random effect must be compared with the fitted SD of that log-clearance random effect, not with a clearance SD in different units.

For epsilon shrinkage, use residuals calculated after conditioning on the individual predictions. In the examples, these are unstandardized residuals in the same units as the observations, and the model has a constant additive residual SD. The residual SD is therefore the appropriate denominator. If the residual model has a different variance at different observations, the analyst must specify how residuals and their reference scale are defined; a raw spread cannot automatically be compared with one constant \(\sigma\).

The most frequent SD-based convention uses the sample SD, with divisor \(n-1\), for the observed EBEs or residuals. Some calculations instead use the population SD of the finite set, with divisor \(n\). Another convention compares variances directly. These choices are not equivalent, especially for small datasets. Whichever is used, the same convention should be applied consistently and named explicitly.

Why the SD Formula Has a Shrinkage Interpretation

In the Gaussian random-intercept setting, the earlier material on conditional distributions and eta shrinkage establishes the basic relationship between the conditional estimate and the population distribution. A brief derivation clarifies what the SD comparison is intended to summarize. Suppose \(\eta_i\sim N(0,\omega^2)\), and the subject’s information can be represented as a noisy measurement \(z_i=\eta_i+e_i\), where \(e_i\sim N(0,\tau^2)\) independently of \(\eta_i\). The conditional mean is

The conditional estimate is the noisy measurement multiplied by a factor between zero and one. Across repeated subjects under this model, its variance is

Since the population variance of \(\eta_i\) is \(\omega^2\), the ratio of the conditional-estimate variance to the population variance is \(\omega^2/(\omega^2+\tau^2)\). This ratio decreases as the measurement noise \(\tau^2\) increases. It motivates comparing the spread of EBEs with the population random-effect scale. In nonlinear models, however, the Gaussian calculation is an explanatory simplification: finite-sample EBE spreads are also affected by the data, model, estimation, and the choice of SD convention.

The variance-based calculation in that Gaussian example would be \(1-\operatorname{Var}(\hat{\eta})/\omega^2\). It is not the same as \(1-s_{\hat{\eta}}/\omega\). If \(s_{\hat{\eta}}/\omega=r\), then the variance-based expression is \(1-r^2\), whereas SD-based shrinkage is \(1-r\). Reporting one formula while calculating the other changes the result.

Epsilon Shrinkage From Conditional Residuals

Epsilon shrinkage is calculated from the spread of residuals after using individual predictions, relative to the model’s residual-error SD. In this simplified setting, let \(e_{ij}=y_{ij}-\hat{y}_{ij}\), where \(y_{ij}\) is an observation and \(\hat{y}_{ij}\) its conditional prediction for subject \(i\) and observation \(j\). The residuals should be computed consistently under the fitted model and compared with \(\sigma\) in the same units. A smaller residual spread relative to \(\sigma\) yields a larger SD-based epsilon-shrinkage value.

Parameter-Specific Values and Convention Checks

Shrinkage is specific to a random-effect component or residual scale, not a universal property of an entire model. Consider a second random effect for log-volume with five centered EBEs \(-0.12\), \(-0.06\), \(0\), \(0.06\), and \(0.12\), and fitted population SD \(\omega_V=0.2000\). The squared deviations sum to \(0.0360\). The sample SD is \(\sqrt{0.0360/4}=0.0948683\ldots\), so the sample-SD eta shrinkage is

This is \(52.57\%\) under the sample-SD convention, compared with \(28.45\%\) for log-clearance in the preceding example. The comparison is meaningful as a parameter-specific description because each EBE spread was divided by its own fitted population SD. It does not establish that the volume estimates are less accurate for every subject, nor does it isolate the cause of the difference.

A useful audit is to calculate the numerator independently from the listed data, retain unrounded values until the final report, and check that the denominator is on the same scale. Also verify the direction of the result: if the EBE SD is below \(\omega\), SD-based shrinkage is positive; if the two are equal, it is zero. A negative estimate can occur if the observed EBE SD exceeds the fitted \(\omega\), especially in small or noisy samples. It signals a ratio above one under this calculation; it is not a probability and should not be silently truncated to zero.

Assumptions, Diagnostics, and Common Pitfalls

These calculations are most interpretable when EBEs or residuals represent the same modeled component across subjects or observations; the relevant population SD or residual SD is positive and estimated on the matching scale; and the calculation method is stated. The examples use centered data and a constant additive residual SD. In practice, check the actual values, missingness, and model specification before treating a single summary as representative. A few observations can dominate the SD, and a small number of subjects makes the sample estimate unstable.

Residuals within a subject may be dependent, and EBEs share fitted population parameters. Thus a computed shrinkage value is descriptive and is not automatically an independent-sample estimate with a simple uncertainty interval. Conditional residuals can also have structure due to model misspecification, not just shrinkage. Inspect residual plots and individual fits alongside the number; systematic patterns, outliers, or changing residual spread make a one-number summary less informative.

• Using the wrong parameter’s denominator. Divide clearance EBEs by the fitted clearance-random-effect SD, not by the volume or residual SD.
• Mixing SD and variance formulas. \(1-s/\omega\) and \(1-s^2/\omega^2\) are different quantities; label the convention actually calculated.
• Changing the divisor without saying so. With few subjects, using \(n\) rather than \(n-1\) can visibly change the SD and shrinkage.
• Rounding too early. Retain the unrounded square root and ratio, then round the final result. For instance, \(\sqrt{0.5120/4}=0.357770876\ldots\), not exactly \(0.3578\); the corresponding result rounds to \(28.45\%\).
• Calling shrinkage a subject-level error probability. It is a ratio-based summary of spread, not the probability that a particular EBE is wrong or a test of the null hypothesis.

The main alternative to a compact shrinkage summary is to examine subject-level EBEs, conditional residuals, and individual fits directly. Those diagnostics retain patterns that a single SD ratio discards; the shrinkage calculation is useful as a concise scale comparison, not as a substitute for inspection. Its limitations include sensitivity to small samples, fitted denominator values, model misspecification, and convention choice.