Conditional Distributions Behind Empirical Bayes Estimates

From a Population Distribution to a Subject-Specific Distribution

An NLME model does not treat each subject’s random effect as an independently estimated fixed parameter. Instead, it specifies a population distribution for the random effects and a model for how each subject’s observations depend on those effects. Once a subject’s data are observed, both pieces of information matter: the population distribution describes plausible effects before using that subject’s data, while the likelihood describes how compatible each possible effect is with the observed data.

The resulting subject-specific conditional distribution is the object that empirical Bayes estimates summarize. It is not the population distribution itself, and it is not simply a point estimate from fitting the subject in isolation. Its shape can contain important information that a single summary cannot show.

For a continuous random effect, Bayes’ rule gives the conditional density as

The likelihood \(p(y_i\mid\eta_i,x_i,d_i;\psi)\) describes the data model when the random effects are fixed at a candidate value. The prior \(p(\eta_i\mid x_i;\psi)\) describes the random-effects distribution implied by the population model for a subject with covariates \(x_i\). The denominator is the marginal probability density of the observed data under that model; it normalizes the numerator so the conditional density integrates to one.

A common specification is \(\eta_i\sim N(0,\Omega)\), with the covariates affecting the typical individual parameters through the fixed-effects model. More generally, the random-effects distribution may depend on covariates. In either case, the likelihood must be built from the specified observation model, including its residual error structure. The formula does not require observations within a subject to be independent; their dependence must simply be represented correctly in the likelihood.

Empirical Bayes Summaries

In practice, population parameters are estimated from the full dataset. Write their estimates collectively as \(\hat\psi\). An empirical Bayes calculation uses those estimates in place of the unknown population parameters, giving the plug-in conditional distribution \(p(\eta_i\mid y_i,x_i,d_i;\hat\psi)\). The term “empirical Bayes” reflects this combination: a population distribution estimated from data is used to form a subject-specific conditional distribution.

A point estimate can summarize that distribution in different ways. Its conditional mean averages over the distribution, whereas its conditional mode locates a value at which the density is largest. In NLME practice, “EBE” is often used for a conditional mode, but terminology and implementation can differ. A report should state which summary is being used rather than assume the label alone settles the matter.

The conditional distribution also has uncertainty and shape. A mean or mode does not report the distribution’s variance, asymmetry, or possible multiple peaks. Thus, when individual uncertainty matters, a point summary alone is incomplete.

Why the Conditional Mean Is a Squared-Error Summary

The conditional mean is not just an arbitrary center. For a scalar random effect with a finite conditional second moment, it minimizes the conditional expected squared error among all point summaries. To see this, let \(a\) be any proposed estimate and let \(m=\mathrm{E}[\eta_i\mid y_i,x_i,d_i]\). Expand the squared error around \(m\):

The cross-term is zero because \(\mathrm{E}[\eta_i-m\mid y_i,x_i,d_i]=0\). Since \((m-a)^2\geq0\), the expected squared error is minimized at \(a=m\). This result explains one reason to use a conditional mean when squared error is the relevant loss; it does not imply that the mean is always the preferred summary for every purpose.

Covariates and Subject Data Both Matter

The prior distribution can vary with covariates if the model specifies such dependence. Even when the prior random-effects distribution is common across subjects, covariates can affect the likelihood through the typical parameter values and thereby affect the conditional distribution. Consequently, two subjects with similar observations need not have identical conditional distributions if their covariates, doses, sampling times, or other design features differ.

Assumptions, Diagnostics, and Alternatives

A conditional distribution is meaningful only relative to a specified model. Its calculation assumes that the prior random-effects distribution is appropriate for the subject’s covariates and that the likelihood correctly represents the observation process, including residual variability and any dependence among repeated observations. It also assumes that the fitted population parameters are adequate for the purpose of the plug-in calculation. The conditional distribution is not a model-free summary extracted from the subject’s data alone.

Useful checks include verifying that conditional probabilities or densities normalize correctly, comparing numerical calculations under finer integration grids or tighter tolerances, and examining whether the result changes materially under reasonable starting values. A sharply peaked, asymmetric, or multimodal conditional distribution deserves more attention than a single reported point can provide. Sparse or weakly informative observations may leave the conditional distribution broad and strongly influenced by the prior. Unexpected concentration at a parameter boundary, extreme sensitivity to one observation, or substantial disagreement between reasonable numerical methods can indicate model or computational problems; these signs are prompts for investigation, not automatic proof of a specific failure.

A subject-by-subject fit that ignores the population distribution is an alternative in the sense that it uses the subject’s likelihood to estimate individual parameters without this prior information. It can be unstable or poorly determined when an individual has limited data, and it answers a different estimation question. A fully Bayesian analysis instead assigns distributions to population parameters and propagates their uncertainty, rather than simply substituting \(\hat\psi\). That approach can be more comprehensive but requires additional modeling and computation. Empirical Bayes summaries are convenient, but they condition on the fitted population parameters as if those estimates were fixed; they do not automatically include uncertainty in \(\hat\psi\).

A common mistake is to call the population distribution the subject’s EBE distribution, or to interpret an EBE as the subject’s directly observed random effect. The correction is to distinguish the prior from the data-conditioned distribution, then state whether the reported EBE is its mean, mode, or another summary. A further limitation is that a good conditional summary cannot repair a misspecified population or observation model.