Linking Eta Shrinkage to Individual Information

Why Information Determines Eta Shrinkage

An individual’s empirical Bayes estimate (EBE) is pulled toward the population value when that individual’s observations provide little information about the individual random effects. The connection is not simply “more observations means less shrinkage.” What matters is how sensitively the observations respond to each random effect, relative to residual variability and the prior variability of that effect.

For a nonlinear model, this connection is local: sensitivities are evaluated near an individual estimate or another relevant value of the random effects. They describe how much the model-predicted observations would change for a small change in each eta. Combining those sensitivities across observations gives a conditional information matrix. In a local Gaussian approximation, that information determines how much the data reduce uncertainty and, under the usual linear-Gaussian conditions, how much the EBEs vary across individuals.

The Jacobian is evaluated at a specified random-effects value, often the individual EBE. Its entries have units of observation per unit of eta. Information therefore depends on the observation scale and on residual precision as well as on sensitivity. A large derivative alone does not guarantee high information if residual variability is also large.

From Local Information to Shrinkage

First consider one random effect \(\eta\sim N(0,\omega^2)\). Suppose that, locally, the vector of observations is linear in \(\eta\): \(\mathbf{y}=\mathbf{f}_0+\mathbf{g}\eta+\boldsymbol{\epsilon}\), where \(\mathbf{g}\) is the sensitivity vector and \(\boldsymbol{\epsilon}\sim N(\mathbf{0},\mathbf{R})\) is independent of \(\eta\). The scalar information is \(I=\mathbf{g}^{\mathsf T}\mathbf{R}^{-1}\mathbf{g}\). Combining the likelihood with the normal prior gives a conditional variance \(v\) and conditional mean \(\widehat{\eta}\) as follows.

The variance \(v\) is the individual’s remaining conditional uncertainty under this model. The fraction of prior variance removed by conditioning on the data is \(1-v/\omega^2=\omega^2I/(1+\omega^2I)\). That is an uncertainty-reduction measure, not the conventional EBE eta-shrinkage statistic. Eta shrinkage is commonly defined as one minus the across-individual standard deviation of the EBEs divided by the population standard deviation \(\omega\). In this exact linear-Gaussian setting, the distinction can be derived directly.

For the first equality, the marginal covariance of the observations is \(\mathbf{g}\omega^2\mathbf{g}^{\mathsf T}+\mathbf{R}\), because the random-effect and residual contributions are independent. Expanding the quadratic form gives \(\omega^2 I^2+I\). Substituting \(v=\omega^2/(1+\omega^2I)\) then gives the final equality. Thus the population variance of the EBEs is \(\omega^2-v\), and the information-based population eta-shrinkage measure is

This result clarifies the direction of the relationship: as information increases, EBE variability approaches the population variability and eta shrinkage decreases. By contrast, the conditional standard deviation relative to the prior standard deviation is \(\sqrt{v}/\omega=1/\sqrt{1+\omega^2I}\). It measures remaining conditional uncertainty, and is not the same as the EBE-based shrinkage statistic.

Several Random Effects and Parameter-Specific Information

For a vector of random effects, local linearization gives an approximate normal conditional distribution with precision matrix \(\boldsymbol{\Omega}^{-1}+\mathbf{I}\), and covariance matrix

This approximation follows by adding the quadratic term in the log prior, with precision \(\boldsymbol{\Omega}^{-1}\), to the quadratic term in the locally linearized log likelihood, with precision \(\mathbf{J}^{\mathsf T}\mathbf{R}^{-1}\mathbf{J}\). It is exact when the observation model is linear in the random effects with normal errors and known covariance. For a nonlinear model, it is a local approximation.

A simple parameter-by-parameter formula is available when the random effects are independent and the local information matrix is diagonal. For random effect \(k\), with prior variance \(\omega_k^2\) and information \(I_k\), its approximate conditional variance is \(\omega_k^2/(1+\omega_k^2I_k)\). Under the corresponding scalar linear-Gaussian conditions, its information-based eta shrinkage is \(1-\sqrt{\omega_k^2I_k/(1+\omega_k^2I_k)}\). The product \(\omega_k^2I_k\) is dimensionless and compares data information with prior precision. Consequently, shrinkage depends on both the data’s sensitivity to that effect and how much population variability that effect has.

The orthogonality condition in that example matters. If the information matrix has nonzero off-diagonal entries, the data inform combinations of random effects, and a scalar formula based only on its diagonal entries is generally not valid. The full matrix must be used to approximate conditional covariance. A parameter-specific summary then depends on that matrix and the prior covariance together, not just on one diagonal element.

Nonlinear Sensitivity Depends on the Observation Design

In a nonlinear model, sensitivities can change with time, dose, covariate values, and the random-effects value at which they are evaluated. A parameter may have little influence on an early prediction but a large influence later, or vice versa. Moreover, only sensitivities at the observations actually collected contribute directly to the individual information. A prediction can be highly sensitive at an unobserved time without those data providing information at that time.

Diagnostics, Alternatives, and Limits

Information-based reasoning is most useful when the approximation is credible and the calculation is interpreted at the right level. Several checks help identify cases where it may mislead.

• Check the sensitivity calculation. Compare analytic derivatives with finite-difference derivatives at the fitted individual random effects. Large discrepancies can indicate a coding error, a poor step size, or a derivative that changes rapidly over the relevant range.
• Check the residual model. The weighting by \(\mathbf{R}^{-1}\) assumes the residual covariance used in the calculation is appropriate. Treating correlated observations as independent can overstate information; an incorrect residual variance can also distort it.
• Check local linearity. If predicted observations change nonlinearly over the range of plausible eta values, a single Jacobian may not adequately represent the likelihood. Compare predictions or likelihood curvature over that range rather than relying on one derivative alone.
• Inspect matrix conditioning. Nearly redundant sensitivity columns can make the information matrix weak in some directions even when individual sensitivities look large. Unstable inverses or very large conditional variances warn that the local information calculation may be numerically or statistically fragile.
• Separate model-based and empirical shrinkage. The formula above describes a population-level result under a specified local Gaussian model. Finite samples, estimated population parameters, nonlinearity, and the method used to compute EBEs can make observed EBE shrinkage differ from this prediction.

The conditional Hessian of the log likelihood is an alternative local curvature measure. For a nonlinear mean model, it can include terms involving residuals and second derivatives of the predictions; the matrix \(\mathbf{J}^{\mathsf T}\mathbf{R}^{-1}\mathbf{J}\) is the expected information under the stated normal model and is also the Gauss–Newton curvature approximation. When nonlinearity is strong, a numerical conditional likelihood analysis or simulation under the fitted model can be more informative than a single local approximation. These approaches do not remove model dependence, but they can reveal whether the local result is representative.