Information About Random-Effect Correlations

Why Correlations Require Joint Information

Population designs can inform not only the amount of between-individual variability in each model parameter, but also how deviations in different parameters vary together. For example, an association between an individual’s clearance deviation and volume deviation may affect the joint pattern of repeated concentrations. Whether a design can detect that association depends on how its measurements respond to both deviations.

The focus here is the covariance contribution to information about random-effect correlations under the Gaussian approximation to the subject-level likelihood. The central calculation has two steps: differentiate the random-effect covariance with respect to a correlation, then propagate that derivative through the measurement sensitivities to the covariance of the observed data. The result describes expected local information under the assumed model; it does not guarantee that a finite sample will estimate the correlation precisely.

Definition: Let the random effects be \(\eta_i\sim N(0,\Omega)\), where \(\Omega\) is their covariance matrix. For distinct random-effect components \(k\) and \(l\), write their standard deviations as \(\omega_k^{1/2}\) and \(\omega_l^{1/2}\), and their correlation as \(\rho_{kl}\). Thus \(\Omega_{kl}=\rho_{kl}\omega_k^{1/2}\omega_l^{1/2}\). When differentiating with respect to \(\rho_{kl}\), holding the standard deviations and other correlations fixed, \(\Omega_{\rho_{kl}}\) has entries \(\omega_k^{1/2}\omega_l^{1/2}\) at positions \((k,l)\) and \((l,k)\), and zero elsewhere.

For a subject with a fixed design, suppose the approximate observation vector has distribution \(Y_i\sim N(\mu_i,V_i)\), where \(V_i=A_i\Omega A_i^T+R_i\). Here \(A_i\) is the matrix of individual sensitivities, evaluated at the population parameter values, and \(R_i\) is the residual covariance. This expression is the linearized covariance propagation used in the earlier material on approximate population means and covariances. If \(A_i\) and \(R_i\) do not depend on \(\rho_{kl}\), the covariance derivative is

V_{i,\rho_{kl}}=A_i\Omega_{\rho_{kl}}A_i^T.

The derivative describes how a small change in the random-effect correlation changes the covariance among measurements for the same individual. It is not itself an information value. To obtain information, it must be assessed relative to the full covariance \(V_i\), including residual variability.

Gaussian Information From a Covariance Derivative

Assume the Gaussian approximation is adequate at the parameter values of interest, \(V_i\) is positive definite, and the parameter is in the interior of its admissible range. First consider a parameter \(\theta\) that affects \(V_i\) but not \(\mu_i\). Writing \(r_i=Y_i-\mu_i\), the log likelihood is a normal log likelihood, and its score for \(\theta\) is

U_{i,\theta} =-\frac{1}{2}\operatorname{tr}(V_i^{-1}V_{i,\theta}) +\frac{1}{2}r_i^T V_i^{-1}V_{i,\theta}V_i^{-1}r_i.

To see why the expected information has its familiar trace form, set \(z_i=V_i^{-1/2}r_i\), so that \(z_i\sim N(0,I)\), and let \(B=V_i^{-1/2}V_{i,\theta}V_i^{-1/2}\). The score becomes \(\tfrac12(z_i^TBz_i-\operatorname{tr}(B))\). For a standard normal vector and a symmetric matrix \(B\), the variance of \(z_i^TBz_i\) is \(2\operatorname{tr}(B^2)\). Therefore the variance of the score, which is the expected Fisher information for this scalar parameter, is \(\tfrac12\operatorname{tr}(B^2)\). Rewriting the trace gives the covariance contribution below.

I_{i,\theta\theta}^{(V)} =\frac{1}{2}\operatorname{tr} \left(V_i^{-1}V_{i,\theta}V_i^{-1}V_{i,\theta}\right).

For two covariance parameters \(\theta_a\) and \(\theta_b\), the corresponding entry is \(\tfrac12\operatorname{tr}(V_i^{-1}V_{i,\theta_a}V_i^{-1}V_{i,\theta_b})\). If the mean also depends on the parameter, the full Gaussian information has an additional mean contribution, \(\mu_{i,\theta_a}^T V_i^{-1}\mu_{i,\theta_b}\). Thus the displayed covariance expression alone is the full information only when the relevant mean derivatives are zero. The covariance-focused calculations below hold the mean fixed with respect to the correlation parameter.

Worked Example: Propagating a Correlation Derivative

Consider two random effects with covariance matrix \(\Omega=\begin{pmatrix}0.16&0.06\\0.06&0.09\end{pmatrix}\). Their standard deviations are \(0.4\) and \(0.3\), so the correlation is \(0.06/(0.4\times0.3)=0.5\). Suppose two hypothetical measurements have sensitivity matrix \(A=\begin{pmatrix}1&2\\2&1\end{pmatrix}\), and independent residual errors each have variance \(0.10\). We calculate the covariance information for \(\rho_{12}\), holding the standard deviations fixed.

The derivative of the random-effect covariance is \(\Omega_{\rho_{12}}=\begin{pmatrix}0&0.12\\0.12&0\end{pmatrix}\), because \(0.4\times0.3=0.12\). Multiplying out the propagated random-effect covariance gives

A\Omega A^T= \begin{pmatrix}0.76&0.80\\0.80&0.97\end{pmatrix}.

For example, the off-diagonal entry is \([1,2]\Omega[2,1]^T=0.16(1)(2)+0.06(1)(1)+0.06(2)(2)+0.09(2)(1)=0.80\). Adding residual variance to the diagonal gives \(V=\begin{pmatrix}0.86&0.80\\0.80&1.07\end{pmatrix}\). Propagating the correlation derivative gives \(V_{\rho_{12}}=A\Omega_{\rho_{12}}A^T=\begin{pmatrix}0.48&0.60\\0.60&0.48\end{pmatrix}\).

The determinant of \(V\) is \(0.86(1.07)-0.80^2=0.2802\), so \(V^{-1}=\frac{1}{0.2802}\begin{pmatrix}1.07&-0.80\\-0.80&0.86\end{pmatrix}\). The product \(Q=V^{-1}V_{\rho_{12}}\) is

Q=\frac{1}{0.2802} \begin{pmatrix} 0.0336&0.258\\ 0.132&-0.0672 \end{pmatrix}.

This product is not symmetric, so its squared trace must be calculated using the actual product entries: for \(Q_0=\begin{pmatrix}a&b\\c&d\end{pmatrix}\), \(\operatorname{tr}(Q_0^2)=a^2+d^2+2bc\). Here the numerator is \(0.0336^2+(-0.0672)^2+2(0.258)(0.132)=0.0737568\). Consequently, the per-subject covariance information is

I_{i,\rho_{12}\rho_{12}}^{(V)} =\frac{1}{2}\frac{0.0737568}{0.2802^2} \approx 0.4697.

This is expected local information for a dimensionless correlation, under the stated Gaussian approximation and hypothetical design. It is not an estimate of the correlation, nor a statement that an observed data set will have a particular precision.

How Repeated Measurements Reveal Correlations

A correlation affects the joint distribution only when observations carry information about both random-effect components, directly or through their covariance pattern. Measurements at different times can have different sensitivity vectors, creating covariance patterns that distinguish joint random-effect variation from independent residual error. Repeated measurements are useful not merely because there are more data points, but because their sensitivity structure can expose the cross-covariance.

Worked Example: Two Measurements With Distinct Sensitivities

Suppose two hypothetical measurements respond separately to two random effects, so \(A=I\). Let the random-effect variances be \(0.25\) and \(0.36\), their correlation be \(0.4\), and the residual errors be independent with variance \(0.09\). The random-effect covariance is \(0.4(0.5)(0.6)=0.12\), and its derivative with respect to the correlation is \(0.5(0.6)=0.30\).

The observation covariance and its derivative are therefore

V=\begin{pmatrix}0.34&0.12\\0.12&0.45\end{pmatrix}, \qquad V_{\rho_{12}}=\begin{pmatrix}0&0.30\\0.30&0\end{pmatrix}.

The determinant is \(0.34(0.45)-0.12^2=0.1386\), and \(V^{-1}=\frac{1}{0.1386}\begin{pmatrix}0.45&-0.12\\-0.12&0.34\end{pmatrix}\). Multiplication gives \(V^{-1}V_{\rho_{12}}=\frac{1}{0.1386}\begin{pmatrix}-0.036&0.135\\0.102&-0.036\end{pmatrix}\). Its squared trace numerator is \(2(0.036^2)+2(0.135)(0.102)=0.030132\). Hence

I_{i,\rho_{12}\rho_{12}}^{(V)} =\frac{1}{2}\frac{0.030132}{0.1386^2} \approx 0.7843.

The nonzero off-diagonal derivative reflects the way the pair of measurements covaries as the random-effect correlation changes. This design can therefore carry information about the correlation. Whether it can distinguish that correlation from other unknown covariance parameters is a separate question: the calculation holds the other parameters fixed.

Information Depends on Sensitivity, Not Just Replication

A measurement design can have many observations and still carry little information about a particular correlation. If all relevant sensitivities to one random effect vanish, then changing the correlation may leave the observation covariance unchanged. Conversely, a single measurement that combines two random effects can have covariance sensitive to their correlation. The following example contrasts those cases.

Worked Example: Zero Information Versus a Combined Measurement

Use the random-effect variances \(0.25\) and \(0.36\), correlation \(0.4\), and residual variance \(0.09\) from the previous example. First suppose one observation measures only the first random effect: \(Y=\eta_1+\epsilon\). Its variance is \(V=0.25+0.09=0.34\), independent of \(\rho_{12}\), so \(V_{\rho_{12}}=0\). The scalar Gaussian covariance information is \(\tfrac12(V_{\rho_{12}}/V)^2=0\). Repeating this same type of measurement does not create covariance sensitivity to the correlation under this model.

Now suppose one observation measures the sum, \(Y=\eta_1+\eta_2+\epsilon\). Its variance is \(V=0.25+0.36+2(0.12)+0.09=0.94\). Since the covariance between the random effects is \(0.30\rho_{12}\), the derivative of this variance is \(V_{\rho_{12}}=2(0.30)=0.60\). The information is

I_{i,\rho_{12}\rho_{12}}^{(V)} =\frac{1}{2}\left(\frac{0.60}{0.94}\right)^2 \approx 0.2037.

The combined measurement has nonzero local information because its variance changes with the correlation. This does not mean that one combined measurement will estimate the correlation well when other covariance parameters are unknown; it shows only that the design has a nonzero covariance derivative in the specified model.

Assumptions, Diagnostics, and Alternatives

These calculations rely on a positive-definite observation covariance and an admissible random-effect covariance. For a two-effect correlation parameter, the interior condition is \(-1<\rho_{kl}<1\), with positive random-effect variances. The Gaussian approximation to the subject-level likelihood must also be reasonable for the design and parameter values considered. If the approximation is poor, the information it implies may not reflect the information in the nonlinear mixed-effects likelihood.

Useful checks before interpreting a result include:

Check covariance propagation. Verify that \(A_i\Omega A_i^T\) is symmetric and that each off-diagonal entry uses both relevant sensitivity vectors. A single incorrect cross-product can substantially alter the information.
Check the derivative. With standard deviations held fixed, the derivative of \(\Omega_{kl}\) with respect to \(\rho_{kl}\) is \(\omega_k^{1/2}\omega_l^{1/2}\), not one. Confirm the propagated derivative by a small finite difference of \(V_i\).
Check positive definiteness. If \(V_i\) is singular or nearly singular, its inverse may not exist or may be numerically unstable. A correlation at its boundary can also violate regularity conditions underlying ordinary Fisher-information calculations.
Check the measurement pattern. If \(V_{i,\rho_{kl}}=0\), the covariance contribution for that subject is zero. A large sample of subjects with the same uninformative pattern does not repair that zero derivative.
Check the scope of the formula. If the approximate mean depends on the correlation, include its mean-information contribution as well. Do not describe covariance-only information as the full Gaussian information in that case.

Expected Fisher information averages over possible observations under the model and is useful for local design comparisons. It differs from observed information, which depends on the realized data. It also differs from directly fitting the likelihood and inspecting the resulting uncertainty: likelihood-based estimation can reveal skewness, boundary behavior, or weak identification that a local Gaussian information calculation does not capture. When the approximation or regularity conditions are doubtful, evaluating the full model likelihood or simulation-based operating characteristics is a relevant alternative, at greater computational cost.

A common mistake is to assume that repeated sampling automatically reveals random-effect correlations. Repetition alone is not enough: the design must produce observations whose covariance changes with the correlation, and that change must be distinguishable from other covariance effects. Another frequent error is to treat \(V^{-1}V_{\rho}\) as symmetric. Even though \(V^{-1}\) and \(V_{\rho}\) are symmetric individually, their product need not be; use the trace expression or multiply matrices explicitly rather than applying a formula that assumes symmetry.

Key takeaway: Information about a random-effect correlation is driven by the covariance derivative \(A_i\Omega_{\rho_{kl}}A_i^T\), weighted by the inverse observation covariance. Measurements that jointly reveal the relevant random effects can create this derivative, while measurement patterns insensitive to either component may provide no covariance information about their correlation.

Check Your Understanding

Answer each question, then compare with the solutions below.

Two random effects have standard deviations \(0.5\) and \(0.2\). What are the two nonzero entries of \(\Omega_{\rho_{12}}\) when differentiating with respect to their correlation?
For a scalar Gaussian observation with variance \(V=0.50\) and derivative \(V_{\rho}=0.10\), calculate the covariance contribution to information for \(\rho\).
If \(A\Omega_{\rho}A^T=0\), what is the covariance contribution to information for that subject, assuming \(V\) is positive definite?
Why can two measurements with different random-effect sensitivities reveal a correlation that a measurement sensitive to only one random effect cannot reveal through its variance?

Solutions. (1) Each nonzero entry is the product of the standard deviations, \(0.5(0.2)=0.10\), at positions \((1,2)\) and \((2,1)\). (2) The scalar formula gives \(\tfrac12(0.10/0.50)^2=\tfrac12(0.20)^2=0.0200\). (3) The derivative is zero, so \(\tfrac12\operatorname{tr}(V^{-1}0V^{-1}0)=0\). (4) The joint observations can have a cross-covariance that changes with the random-effect correlation, while a measurement involving only one component has variance determined by that component’s variance and residual error, not by its correlation with an unobserved component.