Initial Values for Random-Effect Correlations

Why Initialize Random-Effect Correlations Carefully?

A population model can allow individuals with above-typical clearance to have systematically higher or lower volume as well. That relationship is represented by an off-diagonal element of the random-effects covariance matrix. Its initial value matters because the optimizer must begin from a mathematically admissible covariance matrix, and a poor starting point can hinder estimation even when the model itself is reasonable.

The task here is to choose a defensible starting value, not to infer a correlation from a starting guess. A nonzero initial correlation is a computational input. If the correlation is estimated, its fitted value is determined by the data, model, and estimation procedure; it need not resemble the initial value.

Definition: Let \(\eta_i\) be the vector of individual random effects for individual \(i\), with \(\eta_i\sim N(0,\Omega)\). The diagonal elements of \(\Omega\) are random-effect variances; its off-diagonal element \(\Omega_{jk}\) is the covariance between random effects \(j\) and \(k\). Their correlation is \(\rho_{jk}=\Omega_{jk}/\sqrt{\Omega_{jj}\Omega_{kk}}\), provided both variances are positive. An initial correlation is a proposed starting value for \(\rho_{jk}\), not a fitted result.

For example, with log-scale random effects for clearance and volume, \(\log(CL_i/CL_{\mathrm{typ}})=\eta_{CL,i}\) and \(\log(V_i/V_{\mathrm{typ}})=\eta_{V,i}\), the diagonal elements of \(\Omega\) are variances of the log deviations. The off-diagonal element describes their covariance. A positive correlation means that the two log deviations tend to have the same sign; a negative correlation means they tend to have opposite signs. It does not establish a causal relationship.

From a Correlation Guess to a Covariance Matrix

Correlation is dimensionless, while the covariance must be on the scale of the random effects. For two effects with variances \(\omega_1^2\) and \(\omega_2^2\), a chosen correlation \(\rho_{12}\) implies \(\Omega_{12}=\rho_{12}\omega_1\omega_2\). This conversion is essential: entering a correlation directly as a covariance is generally wrong unless the standard deviations happen to make the numerical values coincide.

\Omega = \begin{pmatrix} \omega_1^2 & \rho_{12}\omega_1\omega_2 \\ \rho_{12}\omega_1\omega_2 & \omega_2^2 \end{pmatrix}

A covariance matrix used for a nonsingular multivariate normal random-effects distribution must be positive definite. For the two-effect matrix above, positive variances and \(-1<\rho_{12}<1\) are necessary and sufficient. Values at exactly \(1\) or \(-1\) make the matrix singular, not positive definite.

Justification: The determinant of the two-effect matrix is \(\omega_1^2\omega_2^2-(\rho_{12}\omega_1\omega_2)^2 =\omega_1^2\omega_2^2(1-\rho_{12}^2)\). With both variances positive, the leading diagonal element is positive, and the determinant is positive exactly when \(|\rho_{12}|<1\). These are the positive-definiteness conditions for a symmetric two-by-two matrix.

Worked Example: A Clearance-Volume Starting Correlation

Suppose a hypothetical model uses log-scale random effects for clearance and volume. The proposed initial standard deviations are \(0.30\) for clearance and \(0.40\) for volume. As a working starting assumption, set their correlation to \(-0.25\). Calculate the covariance and check the matrix.

The initial variances are \(0.30^2=0.09\) and \(0.40^2=0.16\). The covariance is \(\Omega_{CL,V}=(-0.25)(0.30)(0.40)=-0.0300\). Thus

\Omega = \begin{pmatrix} 0.09 & -0.03 \\ -0.03 & 0.16 \end{pmatrix}, \qquad \det(\Omega)=0.09(0.16)-(-0.03)^2=0.0144-0.0009=0.0135.

Both diagonal elements are positive and the determinant is positive, so this initial matrix is positive definite. The negative sign encodes a proposed tendency for above-typical log clearance to accompany below-typical log volume, and vice versa. It is only a starting assumption; this calculation does not show that such a relationship is supported by data.

Positive Definiteness With Three or More Effects

Checking every pairwise correlation separately is not enough once there are at least three random effects. Each correlation may lie between \(-1\) and \(1\), yet the full matrix can still be invalid. The constraints apply jointly to the whole correlation matrix.

For three effects, write the correlation matrix as \(R\), with correlations \(a\), \(b\), and \(c\) in the \(12\), \(13\), and \(23\) positions. Its determinant is \(1+2abc-a^2-b^2-c^2\). In addition to positive diagonal elements and valid two-by-two principal minors, positive definiteness requires this determinant to be positive.

R = \begin{pmatrix} 1 & a & b \\ a & 1 & c \\ b & c & 1 \end{pmatrix}, \qquad \det(R)=1+2abc-a^2-b^2-c^2>0

The corresponding covariance matrix is \(\Omega=DRD\), where \(D\) is diagonal with the random-effect standard deviations on its diagonal. If those standard deviations are positive, then \(\Omega\) is positive definite exactly when \(R\) is: for any nonzero vector \(x\), \(x^\mathsf{T}\Omega x=(Dx)^\mathsf{T}R(Dx)\), and \(Dx\) is nonzero. Thus the joint correlation constraints can be checked before applying the scales.

Worked Example: Checking Three Pairwise Starting Correlations

Consider hypothetical log-scale random effects for clearance, volume, and an absorption parameter. Set the proposed correlations to \(a=0.60\) for clearance-volume, \(b=0.50\) for clearance-absorption, and \(c=0.20\) for volume-absorption.

Each value is between \(-1\) and \(1\). The joint determinant check is \(\det(R)=1+2(0.60)(0.50)(0.20)-(0.60)^2-(0.50)^2-(0.20)^2 =1+0.12-0.36-0.25-0.04=0.47\). The two-by-two principal minors are \(1-0.60^2=0.64\), \(1-0.50^2=0.75\), and \(1-0.20^2=0.96\), all positive. The matrix is therefore positive definite.

Now replace \(c=0.20\) with \(c=-0.90\), leaving \(a=0.60\) and \(b=0.50\). Each pairwise correlation remains within the allowable range, but \(\det(R)=1+2(0.60)(0.50)(-0.90)-0.36-0.25-0.81 =1-0.54-1.42=-0.96\). The matrix is not positive definite and cannot be used as a valid nonsingular covariance starting point. The failure is in the joint combination, not in any one pair considered alone.

Choosing a Defensible Initial Value

A practical choice should be guided by what is known about the model and the information available for the analysis. If there is no credible basis for a nonzero association, zero correlation is a neutral starting assumption. If relevant prior analyses or exploratory information support a direction and plausible magnitude, a modest nonzero value may be worth testing. Avoid treating a convenient or biologically appealing number as established fact.

For \(m\) random effects, define a symmetric correlation matrix with ones on the diagonal, choose all off-diagonal entries jointly, check that it is positive definite, and then scale it by the proposed standard deviations. The standard deviations are initialized as described in the earlier material on between-subject variability. Recheck the assembled \(\Omega\), particularly if values have been changed or copied between model parameterizations.

State the random-effect scale.
Identify which effects are modeled, such as log-clearance and log-volume deviations. A correlation refers to those modeled effects, not automatically to raw clearance and volume.

Choose a working correlation structure.
Use zero when no defensible nonzero starting relationship is available. Otherwise document why a proposed direction and magnitude are plausible enough to test.

Check the full matrix.
Verify positive variances and positive definiteness of the correlation matrix, then form \(\Omega=DRD\). Pairwise bounds alone do not establish validity for three or more effects.

Assess dependence on the seed.
When correlation is estimated, compare reasonable valid starting values and examine convergence and the resulting estimates. Treat materially different outcomes as evidence that the fit may be fragile, not as a reason to select the preferred answer.

Worked Example: Comparing Valid Seeds in a Clearance-Volume Fit

Suppose the same hypothetical model has initial standard deviations \(0.30\) for log clearance and \(0.40\) for log volume. Compare zero correlation with a sensitivity seed of \(0.50\).

For zero correlation, the covariance is \(0(0.30)(0.40)=0\), giving \(\Omega_0=\begin{pmatrix}0.09&0\\0&0.16\end{pmatrix}\), whose determinant is \(0.09(0.16)=0.0144>0\). For the \(0.50\) seed, the covariance is \((0.50)(0.30)(0.40)=0.0600\), giving \(\Omega_{0.50}=\begin{pmatrix}0.09&0.06\\0.06&0.16\end{pmatrix}\), whose determinant is \(0.09(0.16)-0.06^2=0.0144-0.0036=0.0108>0\). Both are admissible initial matrices.

If both fits converge successfully and yield similar final estimates, that is reassuring about sensitivity to these seeds, although it does not prove the model is correct. If one fails or the final estimates differ materially, record the issue and investigate the fitting behavior and model support. Do not interpret the seed of \(0.50\) as evidence that the fitted clearance-volume correlation is \(0.50\).

Diagnostics, Alternatives, and Limitations

Before fitting, inspect symmetry, positive diagonal variances, and positive definiteness. A factorization failure or a software warning about an invalid covariance matrix can indicate an inadmissible starting point. Very large absolute correlations, correlations near the boundary, unstable estimates, or strong dependence of results on modest seed changes deserve attention. A valid starting matrix is necessary, but it does not guarantee that the data can estimate every correlation precisely.

The main alternatives are to start with zero correlations, use informed nonzero starting values, or compare a small set of valid seeds as a sensitivity check. Zero can be simple and transparent but may be a less helpful numerical starting point when a meaningful relationship is expected. Informed seeds can improve initialization but introduce subjectivity. Multiple seeds can reveal fragility but do not replace model assessment or establish which correlation is true.

A covariance structure with selected correlations fixed at zero is different from merely initializing them at zero: in the latter case they can still be estimated if the model specifies them as free parameters. Also, correlations among log-scale random effects are not generally numerically identical to Pearson correlations between the exponentiated individual parameters. Finally, an ill-conditioned or nearly singular matrix may be technically positive definite but still create numerical difficulty. Positive definiteness is a mathematical admissibility condition, not evidence that a parameterization is informative or estimable.

Common Mistake

Wrong reasoning: “Every pairwise correlation is between \(-1\) and \(1\), so the covariance matrix is valid.” The three-effect counterexample above disproves this: its pairwise values all satisfy those bounds, but its determinant is negative. Correction: check the full correlation matrix for positive definiteness, then scale it to form the covariance matrix. A second common error is to report the initial correlation as a model finding; only the fitted result, with its uncertainty and diagnostics, can support an interpretation of the estimated relationship.

Key takeaway: Choose initial correlations as explicit, testable starting assumptions. Convert them to covariances using the random-effect standard deviations, and verify positive definiteness of the complete matrix before fitting.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For random-effect standard deviations \(0.20\) and \(0.50\), what covariance corresponds to an initial correlation of \(0.30\)?
For a two-effect correlation matrix, why is a correlation of exactly \(-1\) not a positive-definite starting value?
Three proposed correlations are \(a=0.40\), \(b=0.30\), and \(c=0.20\). Is the three-effect correlation matrix positive definite?
If a correlation is initialized at zero but is free in the model, must its final estimate remain zero? Explain.

Solutions. (1) The covariance is \((0.30)(0.20)(0.50)=0.0300\), in the product scale of the two random effects. (2) Its determinant is proportional to \(1-(-1)^2=0\), so the matrix is singular rather than positive definite. (3) The determinant is \(1+2(0.40)(0.30)(0.20)-0.40^2-0.30^2-0.20^2=1+0.048-0.16-0.09-0.04=0.758>0\). The pairwise principal minors are also positive, so the matrix is positive definite. (4) No. Zero is only the initial value; a free correlation can move during estimation, and its final estimate depends on the data, model, and fitting procedure.