Constructing the Bayesian NLME Posterior

From the NLME Model to a Posterior

In a nonlinear mixed-effects model, observations are linked through subject-specific random effects, while population parameters describe the distribution of those effects and the typical response. A Bayesian analysis treats both the population parameters and the individual random effects as unknown quantities. Its target is therefore a joint distribution over these unknowns, conditional on the observed data.

The construction begins with the model’s probability densities. The observation model gives the likelihood conditional on the random effects and population parameters. The random-effects model supplies a density for the individual effects. A prior supplies a density for the population parameters. Multiplying these components gives the joint posterior up to a normalizing constant.

A common model specifies individual parameters as \(\phi_i=h(\theta,b_i)\) and the conditional response mean as \(f(t_{ij};\phi_i)\). The particular function \(f\), the link \(h\), and the residual distribution depend on the application. For the standard factorization below, assume that subjects’ random effects are independent conditional on \(\Omega\), and that observations are conditionally independent given their subject’s random effects and the population parameters. If observations have serially correlated residuals, the observation likelihood must instead use their appropriate joint density.

The numerator has three conceptually distinct parts. The first is the observation likelihood, evaluated at candidate values of the unknowns. The second is the random-effects density. The third is the prior density for the population parameters. The denominator \(p(y)\) is the normalizing constant, also called the evidence for this model and prior specification. It ensures that the posterior density integrates to one over the full support of \(\eta\) and all \(b_i\).

The integral is over every unknown quantity in the joint posterior. In a nonlinear model it is generally not available in closed form. For posterior sampling, it is often enough to evaluate the numerator, because \(p(y)\) does not vary with the candidate values of the unknowns. This does not mean the normalizing constant is zero or irrelevant to every task: it defines the normalized posterior and can matter when comparing models.

Why the Product Has This Form

The factorization follows directly from the chain rule for densities and the stated conditional-independence assumptions. Conditional on \(\eta\) and the random effects, the observation density is a product over observations. Conditional on \(\Omega\), independent subject effects have density \(\prod_i p(b_i\mid\Omega)\). The prior \(p(\eta)\) completes the joint density of parameters and data. Applying Bayes’ rule then divides this joint density by the data density \(p(y)\).

Under the conditional-independence assumptions, \(p(y\mid\eta,b_1,\ldots,b_N)=\prod_{i,j}p(y_{ij}\mid b_i,\eta)\), and \(p(b_1,\ldots,b_N\mid\eta)=\prod_i p(b_i\mid\Omega)\). Dividing this joint density by its integral over \(\eta\) and \(b_1,\ldots,b_N\) gives the posterior above. This argument also shows where the factorization changes if the model changes: dependence among subjects or within-subject observations must be represented by a different joint density rather than by an unjustified product.

Factorization Across Subjects

The next example makes clear that each subject contributes both observation information and a random-effects density. Use a simple random-intercept model with known residual standard deviation \(1\) and known random-intercept variance \(1\). Suppose the two hypothetical observations are \(y_1=1\) and \(y_2=2\), with one observation per subject. Evaluate the numerator at \(\theta=0\), \(b_1=0\), and \(b_2=1\), using the prior \(\theta\sim N(0,4)\). The effects \(b_1,b_2\) are independent \(N(0,1)\).

The Normalizing Constant in a Small Example

Although the normalizing constant is usually not needed to sample from the posterior, it clarifies the distinction between the numerator and a probability density. In the following example, the integral can be evaluated exactly. This is a special case that illustrates the denominator; general nonlinear NLME models do not usually yield such a simple calculation.

Using the Posterior Kernel Correctly

An MCMC algorithm commonly uses the unnormalized posterior kernel \[ p(y\mid\eta,b)\,p(b\mid\Omega)\,p(\eta), \] or its logarithm, rather than evaluating \(p(y)\). This is valid for sampling because the denominator is constant as a function of the unknowns for fixed observed data and a fixed model. Additive log-density constants can also be omitted if they do not depend on any unknown quantity. Terms that depend on an unknown parameter cannot be discarded merely because they look like constants with respect to the data.

For example, with a normal residual model, the likelihood includes both a squared-residual term and a scale-dependent normalization term. If \(\sigma\) is unknown, the factor involving \(\sigma\) must be retained; dropping it changes the posterior for \(\sigma\). Likewise, a term may be constant in the random effects but depend on \(\Omega\), so it must remain when \(\Omega\) is unknown.

A density also depends on the parameterization in which it is defined. If unknowns are transformed before the posterior is evaluated, the density must be expressed on the transformed scale with the appropriate change-of-variables Jacobian. For example, a density specified for a positive standard deviation is not automatically the same density when expressed in terms of its logarithm. The model’s support and the prior density must be handled consistently.

Checks, Alternatives, and Common Errors

Before using a posterior expression as a computational target, check that each component matches the stated model. The likelihood should use the observed values and the conditional mean or distribution implied by \(f\) and \(h\). The random-effects contribution should have one factor per subject only when the effects are conditionally independent. Every unknown population parameter should appear in the prior, and its support must agree with the model constraints. The resulting kernel should be nonnegative and integrable for a proper posterior.

A useful numerical check is to compute the log posterior as the sum of separate log-likelihood, log-random-effects, and log-prior contributions. Evaluating these pieces separately makes it easier to detect a missing subject, a duplicated prior, an incorrect variance, or an omitted parameter-dependent normalizing term. If the posterior is improper or the likelihood is zero throughout the prior-supported region, a normalized posterior does not exist. Neither issue is repaired by running an MCMC algorithm longer.

The Bayesian joint posterior is not the same object as the likelihood used for maximum likelihood estimation. A likelihood contains the observation density viewed as a function of unknown parameters; it does not itself include the parameter prior. A maximum a posteriori estimate maximizes the posterior density, so its objective includes prior contributions as well as the likelihood and random-effects density. The joint posterior also differs from a posterior that has integrated out some unknowns: here the individual random effects are explicitly included among the unknowns.