Uncertainty in Random-Effect Correlations

Why Correlations Need Careful Uncertainty Summaries

In a nonlinear mixed-effects model, correlations among random effects describe how subject-specific deviations vary together. For example, a correlation between an individual’s random-intercept deviation and random-slope deviation describes their standardized linear association in the model’s random-effects distribution. It does not, by itself, establish a causal relationship or describe a correlation between observed outcomes.

In Bayesian estimation, a correlation is uncertain: each posterior draw gives a possible value, conditional on the model, data, and prior. A single posterior mean or median can conceal substantial uncertainty. Moreover, if one of the corresponding random-effect variances is near zero, the data may contain little information about the correlation, even when its numerical posterior summary appears precise. Interpretation therefore requires both marginal summaries and attention to the joint correlation structure.

Model Definition and Posterior Summaries

Definition: Let \(\boldsymbol{\eta}_i\) be the vector of random effects for subject \(i\), with mean zero and covariance matrix \(\Omega\). For random effects \(a\) and \(b\), their standard deviations are \(\sigma_a\) and \(\sigma_b\), and their correlation is \(\rho_{ab}=\Omega_{ab}/(\sigma_a\sigma_b)\), provided both standard deviations are positive. Equivalently, \(\Omega_{ab}=\rho_{ab}\sigma_a\sigma_b\). The correlation is dimensionless and lies between \(-1\) and \(1\).

A common model assumption is that \(\boldsymbol{\eta}_i\) follows a multivariate normal distribution with covariance matrix \(\Omega\). This assumption concerns the distribution of the random effects; the observation model and the rest of the NLME specification determine how those effects relate to the data. Write \(\Omega\) in terms of standard deviations and a correlation matrix \(R\) as

\Omega = D R D,\qquad D=\operatorname{diag}(\sigma_1,\ldots,\sigma_q).

For posterior draw \(m\), the correlation is \(\rho_{ab}^{(m)}\). The draws must respect the constraints on a correlation matrix: its diagonal entries are one, each pairwise correlation is between \(-1\) and \(1\), and the matrix is positive semidefinite. A marginal posterior summary uses the draws for one correlation alone. For example, the posterior median describes the 50th percentile of that marginal posterior distribution. An equal-tail 90% credible interval uses its 5th and 95th percentiles; conditional on the model and prior, the interval contains 90% of the posterior probability for that correlation.

Worked Example: Summarizing a Marginal Posterior

Suppose ten hypothetical posterior draws for the correlation between a random intercept and a random slope, sorted from smallest to largest, are \(-0.62,-0.45,-0.31,-0.18,-0.07,0.04,0.16,0.29,0.41,0.58\). For this small illustration, use the nearest-rank empirical quantile: the quantile at probability \(p\) is the ordered draw at rank \(\lceil 10p\rceil\).

The sample median is the average of the fifth and sixth ordered draws: \[ (-0.07+0.04)/2=-0.015. \] The empirical 80% equal-tail interval uses ranks \(\lceil 10(0.10)\rceil=1\) and \(\lceil 10(0.90)\rceil=9\), so it is \([-0.62,0.41]\). The median is close to zero, but the interval covers both negative and positive values. These few draws are only a hand-calculation illustration; an actual analysis should use enough well-mixing posterior draws to estimate quantiles reliably.

In context, the example does not provide strong posterior evidence for a particular direction or magnitude of the random-intercept/random-slope correlation: values on both sides of zero remain plausible under the fitted model and prior. The 80% interval is a posterior probability statement, not a claim that repeated intervals would contain a fixed parameter 80% of the time.

Why Small Variance Components Weaken Correlation Information

A correlation is standardized by both random-effect standard deviations. When a standard deviation is near zero, a given change in correlation can correspond to a very small change in covariance. For a pair of random effects,

\Omega_{ab}=\rho_{ab}\sigma_a\sigma_b,\qquad \frac{\partial \Omega_{ab}}{\partial \rho_{ab}}=\sigma_a\sigma_b.

Thus, with \(\sigma_a\) close to zero, changing \(\rho_{ab}\) over much of its allowed range may have little effect on the covariance matrix. If \(\sigma_a=0\) exactly, the covariance between effect \(a\) and another effect is zero regardless of \(\rho_{ab}\); the correlation formula itself is undefined because its denominator is zero. Near, but not at, zero, the correlation remains mathematically defined, but the likelihood may be relatively insensitive to it. This is a reason for caution, not a rule that every small variance automatically makes its correlation unidentifiable: information also depends on the data, design, and other model features.

For example, if \(\sigma_a=0.02\) and \(\sigma_b=0.50\), then \(\sigma_a\sigma_b=0.010\). A change in correlation from \(-0.5\) to \(0.5\) changes the implied covariance from \(-0.005\) to \(0.005\), a difference of \(0.010\). The small scale of this covariance change can make it difficult for the data to distinguish correlation values.

Worked Example: Reading Correlation Alongside Variances

Suppose a hypothetical NLME analysis gives posterior medians \(\sigma_a=0.03\) and \(\sigma_b=0.40\), with a 90% credible interval for \(\rho_{ab}\) of \([-0.85,0.80]\). The posterior median correlation is \(0.10\). At that median, the implied covariance is \[ 0.10(0.03)(0.40)=0.0012. \] At the interval endpoints, keeping the two standard deviations fixed at their medians for this calculation, the corresponding covariances are \[ -0.85(0.03)(0.40)=-0.0102,\qquad 0.80(0.03)(0.40)=0.0096. \]

These endpoint calculations illustrate the scale of covariance associated with the correlation range; they are not a 90% credible interval for covariance, because they hold the standard deviations fixed rather than using their joint posterior draws. The wide correlation interval means the data and model leave substantial uncertainty about the direction and magnitude of association. The small \(\sigma_a\) is a reason to investigate weak information, not proof on its own. Check the posterior draws of both standard deviations and the correlation, and examine whether the correlation summary is stable and well sampled.

Marginal Intervals Do Not Describe the Joint Correlation Matrix

For three or more random effects, each pairwise correlation has a marginal posterior distribution, but the correlations in any single posterior draw must form a valid correlation matrix. Marginal credible intervals summarize each entry separately. They do not show how entries vary together, nor do they imply that every combination of values picked from those intervals is possible.

For three random effects, write the pairwise correlations as \(a=\rho_{12}\), \(b=\rho_{13}\), and \(c=\rho_{23}\). A necessary and sufficient positive-semidefinite condition for their \(3\times3\) correlation matrix is

1+2abc-a^2-b^2-c^2\geq 0, \qquad -1\leq a,b,c\leq 1.

This determinant condition is in addition to the pairwise bounds. It shows why three individually plausible values need not constitute a plausible joint draw. Joint posterior draws retain the dependence and constraints among correlations; separate marginal intervals do not.

Worked Example: A Combination Allowed Marginally but Invalid Jointly

Consider four hypothetical posterior draws of \((a,b,c)\): \((0.9,0.9,0.9)\), \((0.9,-0.9,-0.9)\), \((-0.9,0.9,-0.9)\), and \((-0.9,-0.9,0.9)\). Each draw has determinant \[ 1+2(0.9)(0.9)(0.9)-(0.9)^2-(0.9)^2-(0.9)^2 =1+1.458-2.43=0.028, \] so each defines a positive-definite correlation matrix. The marginal draws for each of \(a,b,c\) take both values \(-0.9\) and \(0.9\); using nearest-rank quantiles, each marginal 50% interval is \([-0.9,0.9]\).

Choosing \(a=0.9,\ b=0.9,\ c=-0.9\) from those marginal ranges gives \[ 1+2(0.9)(0.9)(-0.9)-(0.9)^2-(0.9)^2-(-0.9)^2 =1-1.458-2.43=-2.888. \] The negative determinant means that this combination is not a valid correlation matrix. The example shows why marginal intervals should not be read as a menu of values that can be combined freely. In practice, report marginal summaries for clarity, and use joint draws or plots when relationships among several correlation parameters matter.

Practical Checks, Alternatives, and Limitations

A careful report connects each correlation summary to its random effects and includes the uncertainty, not just a point estimate. Useful checks include:

Inspect the marginal posterior distribution and interval for each correlation. A broad or strongly asymmetric distribution can make a single center summary unrepresentative.
Review the posterior distributions of the corresponding standard deviations. A standard deviation concentrated near zero raises concern that its correlation may be weakly informed.
For several correlations, inspect joint posterior draws or pair plots and confirm that the reported draws represent valid correlation matrices.
Assess sampling quality using trace plots, rank-normalized \( \hat{R} \), effective sample size, and Monte Carlo standard errors. These checks address whether the posterior has been sampled adequately; they do not establish that the model is scientifically appropriate.

A point estimate alone is compact but hides uncertainty. A marginal credible interval adds uncertainty for an individual correlation, but not dependence among several correlations. A joint posterior summary or plot preserves that dependence, although it can be harder to communicate. If a random-effect correlation is not well supported, one alternative is to fit a model without that correlation, when scientifically justified, and compare the resulting model’s implications. Such a change is a modeling decision, not a way to turn an uncertain posterior correlation into a known zero.

The main limitation is that posterior uncertainty is conditional on the specified model and prior. A narrow interval does not guarantee that the correlation is well identified by the likelihood, and a broad interval does not diagnose a single cause. Correlations near their bounds or unstable estimates may warrant additional investigation, but this tutorial’s summaries alone cannot determine whether the cause is sampling difficulty, model structure, or limited information in the data.

Common pitfall: “The posterior median correlation is 0.8, so the two random effects are strongly associated and the result is well determined.” The correction is to inspect the posterior interval and sampling quality, and to check the associated standard deviations. A high median can coexist with substantial uncertainty or a near-zero variance that leaves the correlation weakly informed.

Check Your Understanding

Answer each question, then compare with the solutions.

If \(\sigma_a=0.10\), \(\sigma_b=0.30\), and \(\rho_{ab}=-0.4\), what is the covariance \(\Omega_{ab}\)?
A 90% posterior credible interval for a correlation is \([-0.70,0.20]\). What does this interval say, and what does it not say?
Why can a correlation be weakly informed when one associated random-effect standard deviation is close to zero?
Three pairwise correlations each have marginal intervals that include \(0.9\). Does that guarantee that \((0.9,0.9,0.9)\) is a valid joint draw? Explain using the determinant condition.

Solutions. (1) \(\Omega_{ab}=(-0.4)(0.10)(0.30)=-0.012\), in the product units of the two random effects. (2) Conditional on the model and prior, 90% of posterior probability for the correlation lies in the interval. It does not mean there is a 90% frequentist probability that a fixed parameter is inside this particular interval, and the interval includes both negative and positive values. (3) Since \(\Omega_{ab}=\rho_{ab}\sigma_a\sigma_b\), small \(\sigma_a\) can make the covariance change little across correlation values, reducing the likelihood’s sensitivity to the correlation; this is a warning to investigate, not a certainty. (4) No. For \((0.9,0.9,0.9)\), the determinant is \(1+2(0.9)^3-3(0.9)^2=0.028\), which is positive, so this particular combination is valid. But individual intervals containing \(0.9\) do not establish that the three values occur together in posterior draws; other combinations can violate the determinant condition.