Population Information and Empirical Bayes Shrinkage

Two Different Questions About Information

In a mixed-effects study, data can tell us about the population and about particular individuals. These are related questions, but they are not the same. Population information describes how precisely the full study can estimate shared parameters, such as a population mean or a random-effect variance. Individual empirical Bayes (EB) precision describes how concentrated the model-based estimate of one subject’s latent random effect is, conditional on the population model.

The distinction matters in design. Adding subjects can improve estimation of population parameters even if each subject contributes little information about their own random effect. Repeating measurements on a subject can sharpen that subject’s EB estimate, but it does not necessarily improve every population parameter at the same rate. The calculations below separate these targets in a Gaussian random-intercept model.

A Random-Intercept Model

Suppose subject \(i\) has \(m\) measurements, all made under the same condition. Let \(Y_{ij}\) be measurement \(j\) on subject \(i\), in the measurement’s units. Consider

Here \(\beta\) is the population mean, \(b_i\) is subject \(i\)’s deviation from that mean, and \(\varepsilon_{ij}\) is residual error. Assume random effects are independent across subjects, residual errors are independent across observations and subjects, and the random effects and residual errors are mutually independent. For the calculations, \(\sigma^2\) and \(\omega^2\) are positive and treated as known when calculating EB precision. For population Fisher information, they are the true local values at which information is evaluated.

After integrating out \(b_i\), the vector of a subject’s \(m\) observations has a multivariate normal distribution:

The off-diagonal covariances are \(\omega^2\): two measurements on the same subject share the random intercept. This marginal distribution is the basis for population information. By contrast, EB estimation conditions on the measurements of one subject and combines those data with the random-effect distribution.

Individual EB Estimate and Conditional Precision

For known \(\beta,\omega^2,\sigma^2\), the subject’s data provide the sample mean \(\bar{Y}_i\). The likelihood for \(b_i\) is proportional to a normal density centered at \(\bar{Y}_i-\beta\), with variance \(\sigma^2/m\). Combining it with the normal distribution \(N(0,\omega^2)\) gives a normal conditional distribution for \(b_i\):

The conditional mean is the EB estimate in this known-parameter model. It is a shrunken version of the subject’s raw deviation \(\bar{Y}_i-\beta\). The shrinkage factor is \(\omega^2/(\omega^2+\sigma^2/m)\), between zero and one. The conditional precision, the reciprocal of the conditional variance, is \(1/\omega^2+m/\sigma^2\). It includes information from both the population distribution of random effects and the subject’s own measurements.

In practical empirical Bayes estimation, population parameters are generally estimated from the study and then substituted into these calculations. The resulting conditional variance is therefore a plug-in precision: it treats estimated population parameters as if they were known. It does not, by itself, include uncertainty in those estimates.

Population Fisher Information in the Same Model

For \(N\) independent subjects with the same \(m\), population information is obtained from the marginal likelihood of their observation vectors. For the fixed mean \(\beta\), the Fisher information from one subject is \(\mathbf{1}^{\mathsf T}V^{-1}\mathbf{1}\). Since \(V\mathbf{1}=(\sigma^2+m\omega^2)\mathbf{1}\), it follows that \(V^{-1}\mathbf{1}=\mathbf{1}/(\sigma^2+m\omega^2)\). Therefore,

This is information about the shared population mean, not about any one \(b_i\). The denominator reflects the covariance among repeated observations: extra measurements on one subject share the subject-level random intercept.

The marginal information about \(\omega^2\) is also distinct from EB precision. For a multivariate normal observation with mean not depending on \(\omega^2\), and covariance derivative \(J=\partial V/\partial\omega^2=\mathbf{1}\mathbf{1}^{\mathsf T}\), the covariance-parameter Fisher information is

To see the simplification, \(V^{-1}\mathbf{1}=\mathbf{1}/(\sigma^2+m\omega^2)\) implies \(V^{-1}J=\mathbf{1}\mathbf{1}^{\mathsf T}/(\sigma^2+m\omega^2)\). Squaring this rank-one matrix and taking its trace gives \(m^2/(\sigma^2+m\omega^2)^2\); the Fisher-information formula contributes the factor \(1/2\). Independent subjects contribute additive information, so the total is \(N\) times the per-subject expression. The units are inverse squared units of \(\omega^2\), as required for information about a variance parameter.

These formulas rely on the stated Gaussian model and a balanced design with common \(m\). They describe expected local curvature of the marginal log likelihood. They are not the realized curvature for every possible data set, nor do they guarantee that a variance-component estimate will behave like a normal estimator, especially near the boundary \(\omega^2=0\).

Why Design Can Affect the Two Quantities Differently

A useful comparison holds the total number of measurements fixed while changing how they are allocated. More subjects provide more independent realizations of the random effect; more repeated observations per subject reduce residual noise in that subject’s mean. Those changes enter population information and EB precision through different expressions.

Interpretation, Diagnostics, and Limitations

Population information and EB precision should not be compared as if they were interchangeable versions of one quantity. Population information is a matrix (or a component of a matrix) for shared parameters, based on the marginal distribution across subjects. EB precision is conditional on one subject’s data and the assumed population distribution. Its value is strongly affected by the amount and timing of that subject’s data, while population information also depends on the number and distribution of subjects.

A useful diagnostic is to inspect the ingredients of the calculation against the design and fitted model. For the random-intercept example, check that repeated measurements are genuinely comparable, that residual errors are reasonably represented by the assumed covariance, and that the random-intercept distribution is plausible. If residual errors are correlated but treated as independent, the term \(m/\sigma^2\) can overstate the information contributed by repeated observations and make EB estimates appear too precise. If substantial between-subject structure is omitted, the assumed \(\omega^2\) may also fail to represent the actual heterogeneity.

A common warning sign is a large change in EB estimates or their conditional variances after modest changes to the assumed \(\omega^2\), \(\sigma^2\), or fixed effects. This may indicate that an individual has limited data and is being strongly guided by the population distribution. Conversely, a precise-looking conditional EB variance does not establish that the population variance was estimated precisely: that requires information across subjects. Check both levels, and remember that a plug-in EB variance omits uncertainty in the estimated population parameters.

Alternative summaries answer different questions. A subject-specific likelihood estimate uses that subject’s data without the population shrinkage distribution; it can be unstable when the subject has few observations. A population marginal-likelihood analysis estimates shared parameters from all subjects and is the relevant basis for population information. A full Bayesian analysis can represent uncertainty in both population parameters and individual effects jointly, rather than treating estimated population parameters as fixed. These approaches are not interchangeable: the appropriate one depends on whether the target is a population parameter, an individual effect, or their joint uncertainty.

The formulas here are a transparent reference case, not a universal design rule. Unbalanced sampling, covariates, nonlinear models, non-Gaussian outcomes, estimated nuisance parameters, and more complex random effects change the information calculations. In particular, the displayed EB variance is exact only for the stated Gaussian random-intercept model with known parameters. In applications it is often a conditional approximation based on fitted population parameters.