Profiling Correlations and Covariance Parameters

Why Correlation Profiles Need Special Care

Random-effect correlations can describe whether individual deviations in one model parameter tend to accompany deviations in another. For example, a positive correlation between individual clearance and volume effects means subjects with above-average clearance effects tend also to have above-average volume effects, under the specified random-effects model. Yet correlations differ from unconstrained parameters: each lies between $-1$ and $1$, and in a model with several random effects, not every collection of pairwise correlations is compatible with a valid covariance matrix.

A profile likelihood examines the support for a chosen correlation while allowing other model parameters to adjust. This is useful because the likelihood can be asymmetric, and because the reported correlation may depend strongly on other covariance parameters. Near either boundary, however, the covariance matrix can become nearly singular. An optimizer may then be sensitive to rounding, starting values, or small changes in the data.

Definition and Covariance Constraints

Suppose subject $i$ has a random-effects vector $b_i$ with distribution $N(0,\Omega)$. The model's marginal likelihood integrates over these random effects. Write $\theta$ for the full parameter vector, including fixed effects, residual-error parameters, and random-effects covariance parameters. For random effects $b_j$ and $b_k$, let their standard deviations be $\omega_j$ and $\omega_k$, and their correlation be $\rho_{jk}$. Their covariance is $\Omega_{jk}=\rho_{jk}\omega_j\omega_k$.

Definition: The profile log likelihood for a selected correlation $\rho_{jk}$ is the maximum log likelihood over all remaining model parameters, subject to $\rho_{jk}$ being fixed at a specified value and the resulting covariance matrix being admissible. If $\ell(\theta)$ is the log likelihood, then $$ \ell_p(r)=\max_{\theta:\,\rho_{jk}=r,\ \Omega\ \text{admissible}}\ell(\theta). $$ The profile likelihood ratio statistic at $r$ is $D(r)=2\{\ell(\hat{\theta})-\ell_p(r)\}$, where $\hat{\theta}$ is the unrestricted maximum-likelihood estimate.

For a two-random-effect model, the correlation matrix is $$ R= \begin{pmatrix} 1 & \rho_{12}\\ \rho_{12} & 1 \end{pmatrix}, $$ and positive definiteness requires $-1<\rho_{12}<1$. The endpoints give a singular matrix. With three or more random effects, each pairwise correlation can individually lie in $(-1,1)$ while the full matrix is still invalid. Thus, bounds on the selected correlation alone are not enough: every profiled fit must satisfy the joint matrix constraints.

A convenient representation is $\Omega=DRD$, where $D$ is diagonal with random-effect standard deviations on its diagonal. Profiling a correlation means fixing its entry in $R$, then optimizing the other free parameters, including other correlations and standard deviations as appropriate. It does not mean holding those nuisance parameters at their unrestricted estimates.

Constructing and Interpreting a Profile

Choose a grid of feasible values $r$ for the target correlation. At each value, fix the correlation, maximize the marginal likelihood over the remaining parameters, and record the optimized log likelihood. The profile maximum should occur at, or very near, the unrestricted estimate. The deviance $D(r)$ measures the likelihood loss from imposing that value.

For an interior scalar parameter and under regular likelihood conditions, an approximate 95% profile-likelihood confidence set contains values satisfying $D(r)\leq \chi^2_{1,0.95}=3.841$. The endpoints are where the profile crosses this cutoff. This calibration is not automatically reliable at a boundary or when the likelihood is otherwise nonregular; near $-1$ or $1$, treat it as an approximation and assess sensitivity rather than assuming the usual result applies exactly.

Justification: At a candidate value $r$, the unrestricted fit maximizes $\ell(\theta)$, while the constrained fit maximizes it only among parameter values with $\rho_{jk}=r$. Therefore $\ell(\hat{\theta})\geq\ell_p(r)$ and $D(r)\geq0$. Under regularity conditions, the likelihood-ratio statistic for one constrained scalar parameter is approximately distributed as $\chi^2_1$, which gives the stated cutoff rule. If $r$ is at or near a boundary, the regularity conditions can fail, so this derivation does not guarantee the same reference distribution.

Set the target and feasible grid.
Choose the correlation to profile and candidate values that can occur in a valid covariance matrix.

Refit at each value.
Fix the target correlation and reoptimize all nuisance parameters, checking convergence and covariance admissibility.

Calculate and inspect the profile.
Compute $D(r)$, confirm that it is nonnegative, and check that its minimum is consistent with the unrestricted fit.

Summarize cautiously.
Find the cutoff crossings where regularity is plausible; report boundary proximity and numerical sensitivity when it is not.

Worked Example: A Two-Effect Correlation Profile

Consider a hypothetical NLME model with two random effects on log clearance and log volume. Suppose the unrestricted fit estimates standard deviations $0.30$ and $0.50$, and correlation $\hat{\rho}_{12}=0.60$. The covariance estimate is $0.60(0.30)(0.50)=0.0900$. The estimated covariance matrix is $$ \hat{\Omega}= \begin{pmatrix} 0.0900 & 0.0900\\ 0.0900 & 0.250 \end{pmatrix}. $$ Its determinant is $0.0900(0.250)-(0.0900)^2=0.0144>0$, so it is positive definite.

For illustration, constrained fits give the following log likelihoods. The unrestricted maximum is $-412.0$ at $\rho_{12}=0.60$.

Fixed correlation $r$	Profile log likelihood $\ell_p(r)$	$D(r)=2[-412.0-\ell_p(r)]$
0.00	-414.2	4.40
0.10	-413.65	3.30
0.40	-412.4	0.80
0.60	-412.0	0.00
0.80	-412.8	1.60
0.90	-414.0	4.00

For example, at $r=0.10$, $D=2\{-412.0-(-413.65)\}=3.30$, below 3.841. At $r=0.90$, $D=2\{-412.0-(-414.0)\}=4.00$, above the cutoff. Linear interpolation between the displayed points gives approximate cutoff crossings near $r=0.05$ and $r=0.89$. Thus, under the regular one-parameter approximation, the profile interval is about $(0.05,0.89)$. This interval indicates considerable uncertainty about the magnitude of the correlation; it is not a probability statement about the fixed parameter. The cutoff interpolation is only approximate, so a finer grid and reoptimized fits near the crossings would be appropriate for reporting final endpoints.

Joint Feasibility When There Are Several Correlations

When three random effects are present, a profile for one correlation must respect the other correlations jointly. For a three-by-three correlation matrix with entries $a=\rho_{12}$, $b=\rho_{13}$, and $c=\rho_{23}$, positive definiteness requires positive diagonal entries, positive two-by-two principal minors, and positive determinant. In particular, $$ \det(R)=1+2abc-a^2-b^2-c^2>0. $$ This determinant condition can narrow the feasible range of a profiled correlation well inside its individual bounds.

Worked Example: A Correlation's Feasible Range Depends on the Others

Suppose a hypothetical three-random-effect profile fixes $\rho_{12}=0.80$ and $\rho_{13}=0.70$, while profiling $c=\rho_{23}$. The determinant condition becomes $$ \det(R)=1+2(0.80)(0.70)c-(0.80)^2-(0.70)^2-c^2 =-0.13+1.12c-c^2>0. $$ Equivalently, $c^2-1.12c+0.13<0$. The roots are $$ c=\frac{1.12\pm\sqrt{1.12^2-4(0.13)}}{2} =\frac{1.12\pm\sqrt{0.7344}}{2} \approx 0.132,\ 0.988. $$ The pairwise two-by-two minors are positive because $0.80$, $0.70$, and any feasible $c$ have absolute values below one. Therefore the feasible range for this profile, given the two fixed correlations, is approximately $0.132<c<0.988$, not the full $(-1,1)$.

At $c=0.50$, the determinant is $-0.13+1.12(0.50)-(0.50)^2=0.180>0$, so this candidate is feasible. At $c=0.10$, it is $-0.13+0.112-0.010=-0.028<0$, so that candidate cannot define a positive-definite random-effects covariance matrix. A profile optimizer must exclude it, even though $0.10$ lies between $-1$ and $1$.

Near-Boundary Estimates and Numerical Diagnostics

As $|\rho_{jk}|$ approaches one, the corresponding two-effect covariance block approaches singularity. For standard deviations $\omega_j$ and $\omega_k$, its determinant is $$ \det(\Omega_{jk})=\omega_j^2\omega_k^2(1-\rho_{jk}^2). $$ The factor $1-\rho_{jk}^2$ shrinks toward zero. A small numerical change in the correlation can therefore substantially change the determinant, conditioning, and optimizer behavior. Reporting a rounded estimate of $1.00$ can also obscure whether the fitted value was interior or at a boundary.

Worked Example: Sensitivity Near a Correlation Boundary

Suppose a hypothetical fit estimates random-effect standard deviations $0.40$ and $0.30$, with correlation $0.995$. The covariance is $0.995(0.40)(0.30)=0.1194$, and the determinant of the covariance block is $$ (0.40)^2(0.30)^2(1-0.995^2) =0.0144(0.009975) =0.00014364. $$ If the correlation changes to $0.999$, the determinant becomes $0.0144(1-0.999^2)=0.0144(0.001999)=0.00002879$, about one-fifth as large. If rounded to $1.00$, the determinant is zero and the block is singular. The arithmetic shows why a seemingly small shift in a near-boundary estimate can matter computationally; it does not by itself show that the data support one value over another.

Inspect convergence messages and gradients, repeat constrained fits from multiple starting values, and verify that the fitted covariance matrices remain positive definite. A profile with abrupt jumps, isolated failed fits, or different optimized likelihoods from different starting values may reflect computation rather than a reliable likelihood shape. Repeat the grid more finely near sharp changes and report the actual boundary behavior rather than silently dropping failed points.

Alternatives, Limitations, and Common Mistakes

A Wald interval uses an estimated standard error and a locally quadratic approximation to the likelihood. For a correlation near a boundary, that approximation can extend beyond the allowable range or poorly represent a skewed profile. A profile interval respects the correlation constraint and reoptimizes nuisance parameters, but it can still be unreliable if optimization is unstable, the likelihood is nonregular, or the model provides little information about the correlation. A parametric bootstrap can provide an additional assessment by simulating and refitting under the fitted model; its usefulness depends on the simulation design and successful, appropriately handled refits. Neither alternative repairs a misspecified random-effects distribution.

Correlation estimates can also be sensitive to the random-effects structure and to how much information the observations provide about each random effect. A narrow profile under one fitted model does not establish that the covariance structure is scientifically correct. Interpret the interval as uncertainty conditional on the specified model and likelihood, and consider its numerical reliability separately.

Mistake: profiling while holding nuisance parameters fixed. This is a likelihood slice, not the profile likelihood. Reoptimize the other free parameters at each target value.
Mistake: checking only that the target lies between $-1$ and $1$. In multiple dimensions, the full correlation matrix must be positive definite. Enforce joint covariance constraints at every fit.
Mistake: treating a rounded near-boundary estimate as exactly one. Preserve adequate precision and inspect the fitted matrix and optimization diagnostics before interpreting a boundary result.
Mistake: automatically applying the usual profile cutoff at a boundary. The chi-square calibration relies on regularity. Flag near-boundary inference as approximate and assess it with suitable sensitivity analyses.

Key takeaway: A correlation profile fixes one correlation and reoptimizes the remaining parameters subject to a valid covariance matrix. Its likelihood shape can reveal uncertainty that a local standard error misses, but near plus or minus one, both positive-definiteness constraints and numerical stability require explicit attention.

Check Your Understanding

Try each question, then compare your reasoning with the solutions.

In one sentence, distinguish a correlation profile from a likelihood calculation that fixes both the correlation and all other covariance parameters.
For two random effects with standard deviations $0.20$ and $0.40$ and correlation $-0.50$, calculate their covariance.
In the three-effect example above, is $\rho_{23}=0.10$ feasible when $\rho_{12}=0.80$ and $\rho_{13}=0.70$? Use the determinant condition.
A profile has $D(r)=2.5$ at a candidate interior value. Is it inside the approximate 95% profile confidence set under the regular one-parameter cutoff?
Name two numerical checks that are especially useful when an estimated correlation is close to one.

Solutions. (1) A profile fixes the target correlation but maximizes the likelihood over the remaining parameters; fixing those parameters as well gives only a slice. (2) The covariance is $(-0.50)(0.20)(0.40)=-0.0400$. (3) No. The determinant is $-0.13+1.12(0.10)-(0.10)^2=-0.028<0$, so the matrix is not positive definite. (4) Yes, because $2.5<3.841$, subject to the regularity approximation. (5) For example, check convergence and gradients, repeat the optimization from multiple starting values, and verify positive definiteness of each fitted covariance matrix.