Interpreting Shrinkage and Empirical Bayes Estimates

What an Empirical Bayes Estimate Represents

A population pharmacokinetic model describes both typical parameter values and between-individual variability. For example, an individual’s clearance may be written as \(CL_i=CL_{\mathrm{typ}}\exp(\eta_i)\), where \(CL_{\mathrm{typ}}\) is the typical clearance and \(\eta_i\) is the individual’s random deviation on the log scale. An empirical Bayes estimate (EBE) combines that individual’s concentration data with the model’s population distribution for \(\eta_i\). It is therefore an individual estimate informed by both the data and the population model, not a direct measurement of the individual’s true parameter.

Definition: An empirical Bayes estimate of an individual random effect, written \(\hat{\eta}_i\), is an estimate of that individual’s random effect conditional on their data and on estimated population-model parameters. In many nonlinear mixed-effects implementations it is a conditional mode: the value maximizing the individual’s conditional likelihood multiplied by the population density of the random effect. Some methods report a conditional mean instead. The estimate is “empirical Bayes” because the population distribution is estimated from the data; it is not a fully Bayesian analysis that necessarily propagates uncertainty in the population parameters.

The population distribution acts as a source of information. When an individual’s data are informative, the EBE can be strongly influenced by those data. When the data provide little information about that individual’s parameter, the EBE is pulled toward the population typical value, corresponding to \(\eta_i=0\). This pull is called shrinkage. Shrinkage is a property of the estimate and its information, not, by itself, evidence that the population model is wrong.

A Simple Model for Shrinkage

The mechanism is clearest in a one-parameter normal model. Suppose the data provide a summary \(Y_i\) of an individual’s random effect:

\eta_i \sim N(0,\omega^2), \qquad Y_i\mid \eta_i \sim N(\eta_i,\sigma^2).

Assume individuals’ random effects follow the stated normal distribution, the measurement error is normal with known variance \(\sigma^2\), and the observation is conditionally independent of the random effect except through its mean \(\eta_i\). Treat \(\omega^2\) and \(\sigma^2\) as known for this derivation. The conditional distribution of \(\eta_i\) given \(Y_i=y_i\) is normal. Completing the square in its log density gives:

-\frac{\eta_i^2}{2\omega^2} -\frac{(y_i-\eta_i)^2}{2\sigma^2} = -\frac{1}{2}\left(\frac{1}{\omega^2}+\frac{1}{\sigma^2}\right) \left(\eta_i-\frac{\omega^2}{\omega^2+\sigma^2}y_i\right)^2 +\text{constant in }\eta_i.

The conditional mean and mode are therefore the same in this normal model:

E(\eta_i\mid Y_i=y_i) =\hat{\eta}_i =k y_i,\qquad k=\frac{\omega^2}{\omega^2+\sigma^2}.

Since \(0<k<1\), the estimate lies between the observation \(y_i\) and the population mean zero. A smaller observation-error variance increases \(k\), giving more weight to the individual data; a larger error variance decreases \(k\), giving more weight to the population distribution. The conditional variance is \(\omega^2\sigma^2/(\omega^2+\sigma^2)\), so the estimate’s uncertainty also depends on the amount of individual information.

Worked Example: Shrinkage of an Individual Clearance Effect

Suppose a simplified analysis concerns the log-clearance random effect \(\eta_i\), with \(\omega=0.50\) log units. For one hypothetical individual, the data provide \(y_i=0.60\) log units, with measurement-error standard deviation \(\sigma=0.50\) log units. Use the normal model above.

Calculate the weight on the individual data. The variances are \(\omega^2=0.50^2=0.25\) and \(\sigma^2=0.50^2=0.25\) log units squared, so

k=\frac{0.25}{0.25+0.25}=0.50.

Calculate the EBE. The unshrunk data summary is \(0.60\) log units, and

\hat{\eta}_i=0.50(0.60)=0.30 \text{ log units}.

The estimate is halfway from the data summary to the population mean of zero. On the clearance scale, the estimated multiplicative deviation is \(\exp(0.30)\approx1.35\), or about 35% above typical clearance under this simplified model. This is not a claim that the individual’s true clearance is known to that precision. The conditional variance is \(0.25(0.25)/(0.25+0.25)=0.125\) log units squared, giving a conditional standard deviation of about \(0.354\) log units. The estimate is meaningfully uncertain.

Measuring and Reading Shrinkage

In applied population PK analyses, shrinkage is often summarized across individuals by comparing the spread of EBEs with the estimated population spread of the random effects. One common convention for eta shrinkage is based on standard deviations:

\text{eta shrinkage} =1-\frac{\operatorname{SD}(\hat{\eta}_i)}{\hat{\omega}}.

Here \(\hat{\omega}\) is the estimated population standard deviation for that random effect, and \(\operatorname{SD}(\hat{\eta}_i)\) is the sample standard deviation of the EBEs across individuals. A value closer to one indicates that the EBE distribution is much narrower than the estimated population distribution. Some reports instead use a variance-based measure, \(1-\operatorname{Var}(\hat{\eta}_i)/\hat{\omega}^2\). These are different scales and should not be interchanged without stating the convention. Neither measure is a percentage of individuals whose estimates are wrong, nor does it quantify the uncertainty of one particular EBE.

Worked Example: Interpreting a Shrinkage Summary

In a hypothetical model, the estimated between-individual standard deviation for a random effect is \(\hat{\omega}=0.50\) log units. The standard deviation of its EBEs is \(0.30\) log units. Calculate and interpret the standard-deviation-based eta shrinkage.

1-\frac{0.30}{0.50}=1-0.60=0.40.

The reported shrinkage is 40% under this convention. The EBE distribution has 60% of the model-estimated random-effect standard deviation. This indicates appreciable compression toward zero and cautions against treating the EBEs as if they were the unobserved random effects. It does not mean that each EBE is 40% too small. Under the variance-based convention, the calculation would instead be \(1-(0.30^2/0.50^2)=1-0.36=0.64\), or 64%. The convention must accompany the number.

Shrinkage can also be considered for residuals, often called epsilon shrinkage. Its calculation and interpretation depend on the residual definition and model; it should not be confused with eta shrinkage. In either case, shrinkage summaries are useful warnings about information, not pass/fail thresholds. They can vary by parameter, study design, sampling schedule, and individual.

How Shrinkage Affects Covariate Plots

A plot of EBEs against a covariate is sometimes used to look for a remaining pattern, such as an association between a clearance random effect and body size. Such a plot is not a direct plot of true individual effects: the vertical coordinates have been estimated using both the individual observations and the population distribution. Shrinkage can compress the vertical range and make a trend look weaker. If the amount of information differs across individuals, the compression may differ across the covariate range as well.

The following simple calculation shows attenuation when a covariate effect is present in the underlying effects but not represented in the model. Let \(x_i\) be a centered covariate and suppose the underlying effect is \(\eta_i=\beta x_i+u_i\). In the simplified estimation setting, let \(\hat{\eta}_i=k\eta_i\) for a common \(k\). For the population linear-regression slope of \(\hat{\eta}_i\) on \(x_i\), the relevant assumption is \(E(u_i\mid x_i)=0\) (or, for the population linear-regression slope, that \(u_i\) is uncorrelated with \(x_i\)). Under that assumption:

\operatorname{Slope}(\hat{\eta}_i\text{ on }x_i) = \frac{\operatorname{Cov}(x_i,k(\beta x_i+u_i))} {\operatorname{Var}(x_i)} = k\beta+ k\frac{\operatorname{Cov}(x_i,u_i)}{\operatorname{Var}(x_i)} =k\beta.

Mean-zero \(u_i\) alone is not sufficient: if \(u_i\) is associated with \(x_i\), the covariance term need not be zero. In actual nonlinear mixed-effects analyses, the shrinkage factor can differ by individual, and the simple \(k\beta\) result need not apply. Thus, an EBE plot can help identify a possible pattern, but its apparent slope is not automatically the covariate effect or proof that a covariate is absent.

Worked Example: An Attenuated Covariate Trend

Consider three hypothetical individuals with centered covariate values \(x=(-1,0,1)\). Suppose the underlying effects follow \(\eta_i=\beta x_i+u_i\), with \(\beta=0.40\) log units per covariate unit and \(u=(0.10,-0.20,0.10)\) log units. These deviations have mean zero and are uncorrelated with \(x\), since \(\sum x_i u_i=(-1)(0.10)+(0)(-0.20)+(1)(0.10)=0\). Suppose the model that generated the EBEs does not include the covariate and yields a common shrinkage factor \(k=0.60\).

The underlying effects are \(\eta=(-0.30,-0.20,0.50)\) log units, obtained by calculating \(\beta x_i+u_i\) for each individual. The EBEs are \(k\eta=(-0.18,-0.12,0.30)\) log units. The regression slope through these points is

\frac{\sum_i x_i\hat{\eta}_i}{\sum_i x_i^2} = \frac{(-1)(-0.18)+(0)(-0.12)+(1)(0.30)}{(-1)^2+0^2+1^2} = \frac{0.48}{2} =0.24 \text{ log units per covariate unit}.

The underlying slope is \(0.40\), whereas the EBE slope is \(0.24=0.60(0.40)\). The calculation illustrates attenuation under the stated common-shrinkage and zero-covariance assumptions. It does not establish that every EBE plot will have this exact relationship.

Unequal Information and Subgroup Interpretation

Subgroup comparisons are especially vulnerable when one subgroup has more informative data than another. Information can differ because of sampling times, number of observations, dose histories, or how sensitively the observed concentrations respond to the parameter. A subgroup with less information may have EBEs closer to zero even if its underlying individual effects are not closer to zero. Comparing subgroup averages of EBEs can then mix real differences with differences in estimation precision.

Worked Example: Different Shrinkage Across Two Subgroups

Suppose two hypothetical subgroups have the same population random-effect standard deviation, \(\omega=0.50\) log units. For illustration, both have an individual data summary \(y_i=0.40\) log units. Subgroup A has more informative data, giving \(\sigma^2=0.0625\) log units squared; subgroup B has less informative data, giving \(\sigma^2=0.5625\) log units squared. Apply the normal-model formula separately.

For subgroup A, \(k_A=0.25/(0.25+0.0625)=0.80\), so \(\hat{\eta}_A=0.80(0.40)=0.32\) log units. For subgroup B, \(k_B=0.25/(0.25+0.5625)\approx0.308\), so \(\hat{\eta}_B\approx0.308(0.40)=0.123\) log units. The posterior variances are \((0.25)(0.0625)/(0.3125)=0.0500\) and \((0.25)(0.5625)/(0.8125)\approx0.173\) log units squared, respectively.

The EBEs differ even though the illustrative data summaries are identical, because the information differs. This calculation does not show a true subgroup effect; it shows why an EBE difference alone is not adequate evidence of one. Subgroup exposure questions should be addressed with the population model’s covariate and exposure predictions, together with appropriate uncertainty, rather than by interpreting raw subgroup averages of EBEs as subgroup parameter estimates.

Diagnostics, Alternatives, and Limitations

Before interpreting EBEs, examine how much information the design provides for the parameter and whether that information varies across individuals or covariate groups. Useful warning signs include high eta shrinkage, a narrow EBE distribution relative to \(\hat{\omega}\), many estimates near zero despite substantial estimated between-individual variability, and noticeably different shrinkage across groups. Also consider whether the parameter is identifiable from the individual’s sampling schedule and whether the model has estimated population variability near a boundary. These checks flag limits on interpretation; they do not, by themselves, diagnose the source of model misfit.

EBEs remain useful for summarizing fitted individual behavior and for exploratory displays, especially when data are informative. They are not interchangeable with estimates obtained from each individual’s data alone, which may be highly unstable with sparse sampling. Nor are they substitutes for population-level predictions: the latter use the fitted population model to answer questions about typical or subgroup exposure. When individual-level decisions are required, use an approach that represents individual uncertainty and account for the model’s assumptions; a single EBE can conceal substantial uncertainty.

A common mistake is to see a flat EBE-versus-covariate plot and conclude that the covariate cannot matter. That reasoning ignores shrinkage: if the EBEs are compressed, the plot’s slope can be attenuated, and unequal information can distort it further. The correction is to interpret the plot alongside shrinkage and information, check the covariate relationship in the population model, and avoid treating the EBE trend as an unbiased estimate of a covariate effect. Conversely, a visible EBE trend is not automatically evidence of causation or a basis for adding a covariate; confounding and collinearity can also affect interpretation.

Key takeaway: An EBE is a model-based individual estimate that balances individual data against the population distribution. Shrinkage describes the resulting compression toward the population value. High or unequal shrinkage limits individual and subgroup interpretation, so EBE plots should be read as information-dependent diagnostics rather than as direct observations of true individual parameters.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In the normal model, what happens to \(k\) when the measurement-error variance increases while \(\omega^2\) stays fixed?
A reported standard-deviation-based eta shrinkage is 0.25. What fraction of the estimated population standard deviation is the EBE standard deviation?
Why is mean-zero \(u_i\) alone insufficient to conclude that the slope of \(k(\beta x_i+u_i)\) on \(x_i\) is \(k\beta\)?
Two subgroups have different average EBEs. Give one reason this may occur without a true subgroup difference in the underlying effects.

Solutions. (1) \(k=\omega^2/(\omega^2+\sigma^2)\) decreases as \(\sigma^2\) increases, so the estimate gives less weight to the individual data and more weight to the population mean. (2) By definition, \(0.25=1-\operatorname{SD}(\hat{\eta}_i)/\hat{\omega}\), so \(\operatorname{SD}(\hat{\eta}_i)/\hat{\omega}=0.75\): the EBE standard deviation is 75% of the estimated population standard deviation. (3) The slope includes the additional term \(k\operatorname{Cov}(x_i,u_i)/\operatorname{Var}(x_i)\). Mean zero does not force this covariance to be zero; the result \(k\beta\) requires zero covariance, for example from \(E(u_i\mid x_i)=0\). (4) The subgroups may have different information and therefore different degrees of shrinkage. Their EBE averages can differ even when the underlying effects do not.