Correlated Individual Random Effects in Bayesian Updating

Why Correlation Matters in an Individual Update

An individual’s concentration data may contain substantial information about one pharmacokinetic parameter and little direct information about another. Even so, the posterior for the less-informed parameter can change. The reason is that population random effects may be correlated: observing an individual’s clearance effect, for example, can shift the expected value of that individual’s absorption effect if those effects covary in the population.

This information transfer is a consequence of the joint prior distribution, not evidence that the concentration data directly measured the second parameter. The distinction matters. An indirectly updated parameter can have a posterior mean that moves appreciably while retaining considerable uncertainty, and that update depends on the assumed population covariance structure.

We use a Gaussian model to make the mechanism exact. Let \(\eta_A\) denote the vector of individual random effects directly informed by the data, and let \(\eta_B\) denote the remaining effects. These may be log-scale deviations from typical values. If a structural parameter is written as \(\theta_j=\theta_{\mathrm{pop},j}\exp(\eta_j)\), then a positive value of \(\eta_j\) corresponds to a parameter above its typical value.

Definition: Correlated individual random effects have a joint population distribution with nonzero covariance between at least two effects. In a partition \(\eta=(\eta_A^\mathsf{T},\eta_B^\mathsf{T})^\mathsf{T}\), \(\eta_A\) is directly informed when the individual-data likelihood depends on \(\eta_A\), while \(\eta_B\) is indirectly informed when the likelihood depends on it only through its prior association with \(\eta_A\).

Conditional Distribution and Posterior Update

Suppose the population distribution is multivariate normal with mean zero and covariance matrix partitioned as follows:

\begin{pmatrix} \eta_A\\ \eta_B \end{pmatrix} \sim N\left( \begin{pmatrix} 0\\ 0 \end{pmatrix}, \begin{pmatrix} \Omega_{AA} & \Omega_{AB}\\ \Omega_{BA} & \Omega_{BB} \end{pmatrix} \right).

Assume that the observed data \(y\) depend on \(\eta_A\) but not on \(\eta_B\) once \(\eta_A\) is specified. Formally, \(p(y\mid\eta_A,\eta_B)=p(y\mid\eta_A)\). The conditional distribution of \(\eta_B\) given \(\eta_A\) is then also its conditional distribution given both \(\eta_A\) and \(y\). The standard conditional-normal formula gives:

\eta_B\mid\eta_A,y \sim N\left( \Omega_{BA}\Omega_{AA}^{-1}\eta_A,\; \Omega_{BB}-\Omega_{BA}\Omega_{AA}^{-1}\Omega_{AB} \right).

The posterior for \(\eta_B\) is obtained by averaging this conditional distribution over the posterior uncertainty in \(\eta_A\). Write \(m_A=E(\eta_A\mid y)\) and \(S_A=\operatorname{Var}(\eta_A\mid y)\). Applying the laws of total expectation and total variance yields:

E(\eta_B\mid y)=\Omega_{BA}\Omega_{AA}^{-1}m_A,

\operatorname{Var}(\eta_B\mid y) = \Omega_{BB}-\Omega_{BA}\Omega_{AA}^{-1}\Omega_{AB} + \Omega_{BA}\Omega_{AA}^{-1}S_A\Omega_{AA}^{-1}\Omega_{AB}.

The first variance term is uncertainty about \(\eta_B\) that remains even if \(\eta_A\) were known exactly. The second term is uncertainty propagated from the posterior for \(\eta_A\). Thus, learning about \(\eta_A\) can reduce uncertainty in \(\eta_B\), but it does not generally eliminate it.

For a linear-Gaussian observation model, the update for the directly informed effects is also available in closed form. If \(y=H\eta_A+\epsilon\), with \(\epsilon\sim N(0,R)\) independent of the random effects, and the prior for \(\eta_A\) is \(N(0,\Omega_{AA})\), then:

S_A=(\Omega_{AA}^{-1}+H^\mathsf{T}R^{-1}H)^{-1}, \qquad m_A=S_AH^\mathsf{T}R^{-1}y.

These expressions require a nonsingular prior covariance for \(\eta_A\), a positive-definite observation-error covariance \(R\), and a correctly specified linear-Gaussian likelihood. The conditional-normal relationship can still be used with a non-Gaussian likelihood if the joint prior is normal and the likelihood depends only on \(\eta_A\); in that case \(m_A\) and \(S_A\) may need to be obtained numerically.

Worked Example: Conditional Prior for an Unobserved Effect

Consider a hypothetical population with two standardized log-scale effects: \(\eta_{\mathrm{CL}}\) for clearance and \(\eta_{\mathrm{ka}}\) for the absorption rate constant. Suppose their population distribution is:

\begin{pmatrix} \eta_{\mathrm{CL}}\\ \eta_{\mathrm{ka}} \end{pmatrix} \sim N\left( \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix} 0.36 & 0.24\\ 0.24 & 0.64 \end{pmatrix} \right).

Suppose an individual’s data are informative about \(\eta_{\mathrm{CL}}\), but provide no direct likelihood information about \(\eta_{\mathrm{ka}}\). Before using those data, find the conditional distribution of \(\eta_{\mathrm{ka}}\) at a specified clearance effect of \(0.30\).

The conditional regression coefficient is the covariance divided by the variance of the conditioning effect: \(0.24/0.36=0.6667\). The conditional variance is \(0.64-(0.24)^2/0.36=0.48\). Therefore:

\eta_{\mathrm{ka}}\mid\eta_{\mathrm{CL}}=0.30 \sim N(0.6667(0.30),0.48) =N(0.2000,0.48).

This is a conditional prior statement: among individuals with a clearance effect of \(0.30\), the population model assigns an expected absorption effect of \(0.20\), with conditional variance \(0.48\). It is not yet an individual-data posterior. The relatively large conditional variance also shows that the positive association does not determine the absorption effect exactly.

Worked Bayesian Updates

Worked Example: A Noisy Clearance Observation Updates Absorption

Continue with the hypothetical covariance matrix above. Suppose the available PK data can be summarized by a noisy measurement \(z=0.30\) of the clearance effect: \(z\mid\eta_{\mathrm{CL}}\sim N(\eta_{\mathrm{CL}},0.09)\). This is an illustrative linear-Gaussian likelihood. By construction, the likelihood contains no \(\eta_{\mathrm{ka}}\) term, so the absorption update can occur only through the population correlation.

For the clearance effect, the prior variance is \(0.36\), the observation variance is \(0.09\), and the observation coefficient is 1. The posterior variance and mean are:

S_{\mathrm{CL}} = \left(\frac{1}{0.36}+\frac{1}{0.09}\right)^{-1} =(2.7778+11.1111)^{-1} =0.07200,

m_{\mathrm{CL}} = 0.07200\frac{0.30}{0.09} =0.2400.

The conditional regression coefficient for absorption on clearance is \(0.24/0.36=0.6667\), and its conditional variance is \(0.48\), as calculated above. Thus the marginal posterior mean and variance for absorption are:

m_{\mathrm{ka}}=0.6667(0.2400)=0.1600,

S_{\mathrm{ka}}=0.48+(0.6667)^2(0.07200) =0.48+0.03200=0.5120.

The absorption-effect posterior mean shifts from its population mean of zero to \(0.1600\), even though the likelihood did not directly depend on \(\eta_{\mathrm{ka}}\). Its variance falls from the population variance \(0.64\) to \(0.5120\), but remains substantial. Under the log-scale parameterization, the posterior median for the individual absorption rate is \(\theta_{\mathrm{ka,pop}}\exp(0.1600)\), conditional on this model. The mean on the original parameter scale is not obtained by simply exponentiating the posterior mean of \(\eta_{\mathrm{ka}}\).

Worked Example: Negative Correlation Reverses the Indirect Shift

Now consider a different hypothetical population in which a clearance effect and an effect on another PK parameter have variances \(0.25\) and \(0.49\), and covariance \(-0.21\). Suppose a noisy observation of the clearance effect has value \(z=0.20\) and error variance \(0.0625\). The likelihood again depends only on the clearance effect.

The clearance posterior variance is: \(S_{\mathrm{CL}}=(1/0.25+1/0.0625)^{-1}=(4+16)^{-1}=0.05000\). Its posterior mean is: \(m_{\mathrm{CL}}=0.05000(0.20/0.0625)=0.1600\).

The conditional regression coefficient for the second effect on clearance is \(-0.21/0.25=-0.84\), and its conditional variance is: \(0.49-(-0.21)^2/0.25=0.49-0.1764=0.3136\). Therefore its posterior moments are:

m_B=-0.84(0.1600)=-0.1344, \qquad S_B=0.3136+(-0.84)^2(0.05000)=0.3489.

The indirectly informed effect shifts below its population mean because the covariance is negative and the estimated clearance effect is positive. Its variance decreases from \(0.49\) to \(0.3489\), but the likelihood has not directly measured it. The direction of the shift is determined by the signed covariance, not by a general rule that all updated parameters should move in the same direction.

Worked Example: Two Informed Effects Predict a Third

Suppose the directly informed effects \(\eta_1\) and \(\eta_2\) have independent, unit-variance population distributions. A third effect \(\eta_3\) has variance \(1.50\), with covariances \(0.60\) with \(\eta_1\) and \(-0.40\) with \(\eta_2\). The conditional regression coefficients are therefore \(0.60\) and \(-0.40\); the conditional variance of \(\eta_3\) is \(1.50-0.60^2-(-0.40)^2=0.98\). This is a valid positive conditional variance.

Assume independent hypothetical observations \(z_1=0.40\) and \(z_2=-0.20\), each with error variance \(0.25\), where \(z_j\mid\eta_j\sim N(\eta_j,0.25)\). For either directly informed effect, the posterior variance is \((1+1/0.25)^{-1}=0.20\), and the posterior mean is \(0.20z_j/0.25=0.8z_j\). Hence \(m_1=0.32\), \(m_2=-0.16\), and \(S_A\) is diagonal with entries \(0.20\).

The third effect’s posterior mean is the conditional regression applied to the two posterior means: \(m_3=0.60(0.32)-0.40(-0.16)=0.2560\). Its posterior variance is the conditional variance plus the propagated uncertainty:

$$ S_3=0.98+(0.60)^2(0.20)+(-0.40)^2(0.20) =0.98+0.072+0.032 =1.084. $$

Thus, information in both observed effects changes the posterior for the third, even though its likelihood contribution is absent. The covariance of \(\eta_3\) with the two informed effects after updating is \(0.60(0.20)=0.12\) and \(-0.40(0.20)=-0.08\), respectively. These remaining posterior covariances are consistent with the same conditional relationship and the reduced uncertainty in the informed effects.

Derivation, Diagnostics, and Alternatives

The key variance expression follows directly from the law of total variance. Let \(B=\Omega_{BA}\Omega_{AA}^{-1}\) and \(C=\Omega_{BB}-\Omega_{BA}\Omega_{AA}^{-1}\Omega_{AB}\). Then \(\eta_B\mid\eta_A,y\) has mean \(B\eta_A\) and variance \(C\). Consequently:

\operatorname{Var}(\eta_B\mid y) =E\{\operatorname{Var}(\eta_B\mid\eta_A,y)\mid y\} +\operatorname{Var}\{E(\eta_B\mid\eta_A,y)\mid y\} =C+B S_A B^\mathsf{T}.

Substituting the definitions of \(B\) and \(C\) gives the posterior variance formula above. This shows exactly why the residual conditional uncertainty \(C\) cannot be removed by learning \(\eta_A\) alone. It also shows that if the data become increasingly precise about \(\eta_A\), so that \(S_A\) approaches zero, the posterior variance of \(\eta_B\) approaches \(C\), not generally zero.

In an applied nonlinear PK model, the likelihood may not be a direct measurement of a random effect. The conditional result still applies when the likelihood depends only on \(\eta_A\), but the posterior of \(\eta_A\) may need Laplace approximation, MCMC, or another suitable numerical method. If the likelihood also depends on \(\eta_B\), the factorization used here no longer holds, and the full joint posterior must be updated.

Useful diagnostics focus on whether the assumed information pathway is credible:

Check the covariance structure. Confirm that the estimated or specified covariance matrix is positive definite and that the implied conditional variances are nonnegative.
Inspect posterior sensitivity. Refit or re-evaluate the individual posterior under plausible alternative correlations. A large change in an indirectly informed effect signals dependence on the population covariance assumption.
Check the likelihood’s dependence. Use the structural model and sampling design to establish which parameters the individual data can inform. “Little direct information” is not the same as “no likelihood dependence.”
Assess prior-data compatibility and fit. A large indirect shift alongside poor concentration predictions or strong prior-data conflict should not be treated as a reliable parameter measurement.
Inspect uncertainty as well as the point estimate. A shifted posterior mean may coexist with a broad posterior distribution. Report uncertainty on the random-effect and, where useful, original parameter scales.

An independent-random-effects model sets \(\Omega_{BA}=0\). Under the assumptions above, it produces no indirect shift and no reduction in the marginal uncertainty of \(\eta_B\) from data about \(\eta_A\). This may be appropriate if independence is supported, but imposing it merely for convenience discards potentially relevant population information. At the other extreme, relying on a fixed correlation without sensitivity checks can overstate how much the individual data say about \(\eta_B\).

A joint numerical posterior method can represent non-Gaussian likelihoods and complex parameter dependencies, but it does not remove the need to specify and assess the population covariance model. The closed-form Gaussian update is valuable as an exact calculation under its assumptions and as a check on numerical implementations. It is not a substitute for a correct structural model or sufficient individual data.

Common Mistake and Practical Interpretation

Common pitfall: “The data estimated the absorption effect because its posterior mean changed after observing concentrations.” This is incorrect if the likelihood depended only on clearance. The change is an indirect consequence of the assumed correlation between the effects. State which parameter the likelihood informs, identify the population covariance that transmits information, and retain the conditional uncertainty in the posterior.

Correlated random effects are useful because they allow population-level relationships to contribute to an individual’s estimate when that individual’s data are sparse. Their value is also their limitation: the information transferred to a weakly identified parameter is only as credible as the covariance model and the assumed likelihood structure. In dose-informed decisions, that distinction should remain visible in both uncertainty summaries and sensitivity analyses.

Check Your Understanding

Answer each question, then compare your reasoning with the solution directly below it.

A directly informed effect has posterior mean \(0.25\). An indirectly informed effect has conditional regression coefficient \(0.40\) on the first effect and conditional variance \(0.30\). What is the indirectly informed effect’s posterior mean and variance if the first effect’s posterior variance is \(0.10\)?
Solution: The mean is \(0.40(0.25)=0.100\). The variance is \(0.30+(0.40)^2(0.10)=0.316\).
What happens to the indirect posterior mean if the covariance is zero and the likelihood depends only on the directly informed effect?
Solution: The conditional regression coefficient is zero, so the indirect posterior mean remains at its population mean. Under this model, its marginal variance also remains at its prior variance.
Suppose an indirect effect has conditional variance \(0.22\), and its regression coefficient on the directly informed effect is \(-0.50\). If the directly informed effect has posterior mean \(0.30\) and variance \(0.08\), calculate the indirect posterior mean and variance.
Solution: The mean is \(-0.50(0.30)=-0.150\). The variance is \(0.22+(-0.50)^2(0.08)=0.240\).
If data become arbitrarily precise about \(\eta_A\), must the posterior variance of \(\eta_B\) approach zero?
Solution: No. As \(S_A\) approaches zero, the propagated component \(B S_A B^\mathsf{T}\) approaches zero, but the remaining variance is the conditional covariance \(\Omega_{BB}-\Omega_{BA}\Omega_{AA}^{-1}\Omega_{AB}\). This is generally positive.
The PK likelihood depends on both \(\eta_A\) and \(\eta_B\), rather than only on \(\eta_A\). Can the conditional update above be used unchanged?
Solution: No. In that case \(p(\eta_B\mid\eta_A,y)\) need not equal \(p(\eta_B\mid\eta_A)\). The joint posterior must account for both effects in the likelihood; the displayed indirect-only formulas do not apply unchanged.