Conditional and Marginal Bayesian NLME

Two Views of the Same Hierarchical Model

In a nonlinear mixed-effects model, each subject’s observations depend both on population parameters and on subject-specific random effects. Bayesian inference can keep those effects explicit as unknown quantities, or integrate them out when forming the likelihood for population parameters. These are not different scientific models: they are two related views of the same joint probability model.

The distinction matters because a random effect is not an observed covariate. Conditioning on its value means treating it as part of the unknown state in the joint posterior. Integrating it out instead averages the observation model over the assumed distribution of random effects. The resulting marginal likelihood reflects between-subject variation rather than pretending that each subject effect is known.

Definition: Let \(\theta\) denote population parameters, \(b_i\) the random effect for subject \(i\), and \(y_i\) that subject’s data. The conditional posterior of the random effects given population parameters is \(p(b_1,\ldots,b_N\mid \theta,y)\). The marginal posterior of the population parameters is obtained by integrating all random effects out of the joint posterior: \(p(\theta\mid y)=\int p(\theta,b_1,\ldots,b_N\mid y)\,db_1\cdots db_N\). The corresponding marginal likelihood integrates the random effects out of the sampling model.

Joint, Conditional, and Marginal Distributions

For independent subjects, a standard hierarchical specification factors the joint posterior into a prior for the population parameters, a random-effects distribution, and a conditional observation model. In general notation,

p(\theta,b_1,\ldots,b_N\mid y) \propto p(\theta)\prod_{i=1}^{N}p(b_i\mid\theta)\,p(y_i\mid b_i,\theta).

This expression is the joint posterior: it describes uncertainty in the population parameters and all subject effects together. The conditional posterior of the effects for fixed \(\theta\) is proportional to \(\prod_i p(b_i\mid\theta)p(y_i\mid b_i,\theta)\). The population-parameter posterior instead follows by integrating each subject effect:

p(\theta\mid y) \propto p(\theta)\prod_{i=1}^{N} \underbrace{\int p(b_i\mid\theta)p(y_i\mid b_i,\theta)\,db_i}_{p(y_i\mid\theta)}.

The integral \(p(y_i\mid\theta)\) is subject \(i\)’s marginal likelihood contribution. It averages the conditional likelihood across the random-effects distribution; it does not select a single best-fitting value of \(b_i\). Once draws or estimates for \(\theta\) are available, subject effects can still be inferred from their conditional posterior distributions. Their marginal posterior is a mixture over uncertainty in \(\theta\):

p(b_i\mid y)=\int p(b_i\mid\theta,y)\,p(\theta\mid y)\,d\theta.

Deriving the Random-Intercept Marginal Likelihood

Consider a simple random-intercept model with \(N\) independent subjects and \(m_i\) observations per subject:

y_{ij}=\mu+b_i+\epsilon_{ij},\qquad b_i\sim N(0,\tau^2),\qquad \epsilon_{ij}\sim N(0,\sigma^2).

Assume the random effects are independent across subjects, the residual errors are independent across observations and subjects, and all residual errors are independent of all random effects. Here \(\mu\) is the population intercept, \(\tau^2\) is the between-subject variance, and \(\sigma^2\) is the residual variance. For the derivation below, treat these parameters as fixed. Conditional on \(b_i\), the observations for subject \(i\) are independent normal variables with mean \(\mu+b_i\) and variance \(\sigma^2\).

For a subject with \(m_i=m\), the subject mean is \(\bar y_i=\mu+b_i+\bar\epsilon_i\). Since \(\bar\epsilon_i\sim N(0,\sigma^2/m)\) and is independent of \(b_i\), adding the independent normal terms gives the marginal distribution of the subject mean:

\bar y_i\mid\mu,\tau^2,\sigma^2 \sim N\left(\mu,\tau^2+\frac{\sigma^2}{m}\right).

Thus the variance of a subject mean has two components. The term \(\tau^2\) remains even with many observations per subject; it represents variation between subjects. The term \(\sigma^2/m\) is the residual contribution to the mean and decreases as the number of observations per subject increases.

The same result can be written for the full observation vector. Let \(\mathbf{1}\) be a vector of \(m_i\) ones and \(I\) the identity matrix of size \(m_i\). Integrating the normal random intercept gives

y_i\mid\mu,\tau^2,\sigma^2 \sim N\left(\mu\mathbf{1},\;\tau^2\mathbf{1}\mathbf{1}^{\mathsf T}+\sigma^2I\right).

The off-diagonal covariance is \(\tau^2\): two observations from the same subject share the same random intercept. Treating the observations as marginally independent would discard this dependence and generally misstate the information in the data.

Worked Example: Population Inference From Integrated Subject Effects

Suppose three hypothetical subjects each contribute two measurements. Their data are \((9,11)\), \((11,13)\), and \((13,15)\), so their subject means are 10, 12, and 14. Assume \(\tau^2=2.25\) and \(\sigma^2=1\) are known, and use the prior \(\mu\sim N(10,4)\). Find the marginal posterior for \(\mu\).

Check the model conditions. For this calculation, the subjects are independent, the two residual errors within each subject are independent with variance 1, and the subject intercepts are independent with variance 2.25. The stated normal distributions make each subject mean normal with variance \(2.25+1/2=2.75\). Thus the three subject means are conditionally independent given \(\mu\), each with distribution \(N(\mu,2.75)\).

The prior precision is \(1/4=0.25\), and each subject mean contributes precision \(1/2.75=4/11\). The posterior precision and variance are therefore

\frac{1}{4}+\frac{3}{2.75} =0.25+1.09091 =1.34091,\qquad V_{\mu\mid y}=\frac{1}{1.34091}=\frac{44}{59}\approx0.7458.

The posterior mean is the precision-weighted average of the prior mean and the subject means:

E(\mu\mid y) = \frac{10/4+(10+12+14)/2.75}{1/4+3/2.75} = \frac{2.5+13.0909}{1.34091} \approx11.627.

Consequently, \(\mu\mid y\sim N(11.627,0.7458)\), where the second normal argument is the variance. A 95% posterior credible interval is \(11.627\pm1.96\sqrt{0.7458}\), or approximately \((9.934,13.320)\). Given this model, prior, and data, the posterior probability that \(\mu\) lies in this interval is 0.95. This interval is for the population intercept, not for an individual subject’s intercept.

Conditional Inference for an Individual Effect

For fixed \(\mu,\tau^2,\sigma^2\), the conditional posterior of \(b_i\) combines its normal distribution in the population with that subject’s residual data. Completing the square in the product of the normal prior density for \(b_i\) and the conditional likelihood yields

b_i\mid y_i,\mu,\tau^2,\sigma^2 \sim N\left( \frac{m_i\tau^2}{\sigma^2+m_i\tau^2}(\bar y_i-\mu), \frac{\tau^2\sigma^2}{\sigma^2+m_i\tau^2} \right).

The conditional mean is a shrunken version of \(\bar y_i-\mu\). When the subject has little information, the posterior mean is closer to zero, the population mean of the random effects. When the subject has many precise observations, the data receive greater weight. The posterior variance is smaller than the prior variance \(\tau^2\), provided the data contain information about that effect.

Worked Example: Conditional Posterior Effects for Three Subjects

Use the data and variance assumptions in the preceding example, but first condition on \(\mu=11.627\). For every subject, \(m_i=2\), \(\tau^2=2.25\), and \(\sigma^2=1\). The posterior variance of each effect is

\frac{2.25(1)}{1+2(2.25)} =\frac{2.25}{5.5} =\frac{9}{22} \approx0.4091.

The multiplier applied to each difference \(\bar y_i-\mu\) is \(m_i\tau^2/(\sigma^2+m_i\tau^2)=4.5/5.5=9/11\approx0.8182\). Therefore the conditional posterior means are:

Subject 1: \((9/11)(10-11.627)\approx-1.331\).
Subject 2: \((9/11)(12-11.627)\approx0.3051\).
Subject 3: \((9/11)(14-11.627)\approx1.942\).

Each conditional posterior standard deviation is \(\sqrt{9/22}\approx0.6396\). These are distributions for the subject effects conditional on the selected population intercept and known variances. They are not the effects’ full marginal posteriors, because they omit uncertainty in \(\mu\). To obtain a marginal posterior for an effect, one must also average its conditional posterior over the posterior uncertainty in the population parameters.

Worked Example: Why Conditioning Is Not the Same as Integrating

Keep the same six observations and prior for \(\mu\). Suppose, just for illustration, that we condition on all three random effects being exactly zero. This is a conditional calculation, not an estimate of their actual values. With the effects fixed at zero, the six residual observations have variance 1, so the conditional likelihood precision for \(\mu\) is \(6/1=6\). The posterior variance and mean under this conditioning are

V_{\mu\mid y,b=0}=\frac{1}{1/4+6}=\frac{1}{6.25}=0.16, \qquad E(\mu\mid y,b=0)=\frac{10/4+72}{6.25} =\frac{74.5}{6.25}=11.92.

This conditional posterior, \(N(11.92,0.16)\), differs from the marginal posterior \(N(11.627,0.7458)\) found by integrating the random effects. The conditional calculation acts as though the six observations provide direct information about \(\mu\) with no uncertainty from subject-to-subject variation. The marginal calculation accounts for the shared intercept within each subject and the between-subject variance. Conditioning on a particular value is valid as a conditional statement; treating that value as known when it is not is not a substitute for marginal inference.

Choosing and Interpreting the Two Views

Both approaches are useful, but answer different computational questions. Keeping the effects explicit gives a joint posterior for population parameters and subject-specific effects. Integrating effects gives a posterior for population parameters whose likelihood already accounts for random-effect variation. If individual effects are required after marginal inference, their conditional distributions can be evaluated across population-parameter draws and combined.

View	What is represented	Useful for	Main caution
Conditional or joint	Population parameters and latent subject effects together	Inference about individual effects and the full joint posterior	Conditional summaries depend on the population-parameter values used
Marginal	Population parameters after integrating subject effects out	Population inference that accounts for the random-effects distribution	The integral may be difficult in nonlinear models

In a linear normal random-intercept model, the integral is available analytically. In a nonlinear NLME model, the conditional likelihood may be nonlinear in the random effects, and the marginal integral may not have a closed form. The two formulations remain mathematically connected, but evaluating the marginal likelihood can require numerical integration or other approximations. Explicitly retaining effects avoids directly forming that integral, while creating a higher-dimensional joint inference problem.

A practical diagnostic is to check whether the assumed marginal dependence matches the design and model. In the random-intercept example, observations from one subject have covariance \(\tau^2\); a calculation that treats them as independent marginally contradicts the model. Also check that conditional effect summaries are reported with the population-parameter values or uncertainty on which they depend. A single conditional mean is not automatically a full uncertainty summary.

Mistake: treating a random effect as an observed covariate. Substituting a fitted or assumed value and then using the conditional likelihood as if that effect were known ignores uncertainty. The correction is to retain the effect in the joint posterior or integrate over its distribution.
Mistake: treating conditional and marginal posteriors as interchangeable. \(p(b_i\mid y_i,\theta)\) fixes \(\theta\); \(p(b_i\mid y)\) averages over its posterior uncertainty. State which one is being summarized.
Mistake: assuming integration removes individual information. Integration removes the effects from the population-parameter likelihood, but individual effects can still be recovered through their conditional posterior distributions.

Key takeaway: Conditional inference keeps random effects explicit; marginal inference averages them out under their model distribution. The marginal likelihood and conditional posterior are linked parts of one joint model, and neither should be interpreted as if it were the other.

Check Your Understanding

Answer each question using the stated random-intercept model. Solutions follow the questions.

With one observation per subject, what is the marginal variance of an observation conditional on \(\mu\)?
For two observations per subject with \(\tau^2=1\) and \(\sigma^2=4\), what is the marginal variance of the subject mean?
In the model above, what is the marginal covariance between two different observations from the same subject?
Explain why a posterior for \(b_i\) conditional on one fixed \(\mu\) is not generally the marginal posterior for \(b_i\).

Solutions. (1) With \(m=1\), the marginal variance is \(\tau^2+\sigma^2\), since the observation contains both the random intercept and one residual error. (2) The mean variance is \(\tau^2+\sigma^2/m=1+4/2=3\). (3) The two observations share \(b_i\), so their covariance is \(\operatorname{Var}(b_i)=\tau^2\); independent residual errors contribute no covariance. (4) The conditional posterior fixes \(\mu\), whereas the marginal posterior averages \(p(b_i\mid y_i,\theta)\) over \(p(\theta\mid y)\). If \(\mu\) is uncertain, fixing it generally omits part of the uncertainty in \(b_i\).