Within-Visit Residual Covariance Across Markers

Why Same-Visit Residual Covariance Matters

When several markers are measured on the same subject at the same visit, their values may still move together after accounting for their modeled means and subject-specific random effects. For example, a shared laboratory condition or short-term physiological fluctuation may affect multiple measurements at once. A model that rules out all such residual association can misrepresent the dependence in the data, even if its random-effects covariance is flexible.

The key distinction is between dependence represented by subject-level random effects and dependence left among measurements made together. The earlier material on block covariance structures for marker random effects concerns the former. This tutorial defines the latter and compares common ways to model it.

Model and Assumptions

Suppose \(p\) markers are measured together at visit \(j\) for subject \(i\). Write the \(p\)-vector of observations as \(Y_{ij}\), and use the multivariate linear mixed model

Here \(\mu_{ij}\) is the vector of fixed-effects means, \(b_i\) is the subject-level vector of random effects, and \(Z_{ij}\) maps those effects to the marker vector. Assume \(E(b_i)=0\), \(\operatorname{Cov}(b_i)=G\), and \(E(\epsilon_{ij}\mid b_i)=0\). For the basic within-visit specification, assume

The same \(\Sigma_{\epsilon}\) is used at each visit and for each subject in this homogeneous version. Residual vectors from different subjects are also assumed independent. These are modeling assumptions, not consequences of having repeated measurements. If the residual covariance changes with visit or residuals remain associated across distinct visits, this basic specification is inadequate.

A covariance entry has units equal to the product of the units of its two markers. Its corresponding residual correlation is dimensionless. A positive entry means that, conditional on the model, above-mean residuals for one marker tend to occur with above-mean residuals for the other; it does not by itself imply a causal relationship.

Separating Residual and Random-Effect Covariance

The observed covariance between markers at one visit contains both sources of dependence. Conditional on the random effects, the covariance is \(\Sigma_{\epsilon}\). Marginally, it also includes variation in \(b_i\). By the covariance decomposition,

For two distinct visits \(j\ne k\) on the same subject, the assumed residual independence gives

Thus a same-visit covariance is not automatically a residual covariance: shared random effects also contribute to it. Conversely, cross-marker residual covariance is not a substitute for random effects, which can induce dependence across visits as well as across markers.

Three Covariance Specifications

The covariance specification determines which residual associations the model can represent. For a covariance matrix to be valid, it must be symmetric and positive semidefinite; a positive-definite matrix is generally needed for a nonsingular multivariate normal model. Each proposed parameterization must respect this requirement.

Under independent errors, \(\Sigma_{\epsilon}\) is diagonal. Its \(p\) diagonal entries can differ, so independence does not require equal residual variances. An unstructured matrix has \(p\) variances and \(p(p-1)/2\) distinct off-diagonal covariances. This flexibility can be useful when the number of markers is small and the data support estimating all entries, but it can be unstable when information is limited.

A common parsimonious option is compound symmetry: all markers have a common residual variance and all distinct marker pairs have a common residual correlation. Another option is heterogeneous compound symmetry, which allows marker-specific variances but shares a common correlation. A factor-analytic specification represents covariance through a small number of shared latent factors plus marker-specific residual variances. These structures can reduce the number of free parameters, but their restrictions should be scientifically and empirically defensible.

A Parsimonious Structure and Its Validity

For \(p\) markers, compound symmetry with common residual variance \(\sigma_{\epsilon}^{2}\) and common correlation \(\rho\) is

The diagonal entries are \(\sigma_{\epsilon}^{2}\); off-diagonal entries are \(\rho\sigma_{\epsilon}^{2}\). The eigenvalues are \(\sigma_{\epsilon}^{2}(1-\rho)\), with multiplicity \(p-1\), and \(\sigma_{\epsilon}^{2}\{1+(p-1)\rho\}\), with multiplicity 1. Therefore, for \(\sigma_{\epsilon}^{2}>0\), positive definiteness requires

The lower bound matters: a common correlation below zero is possible, but it cannot be arbitrarily negative. The upper bound prevents perfectly redundant residuals, which would make the matrix singular.

Diagnostics, Comparisons, and Limitations

A useful diagnostic begins with residuals that account for the fitted fixed effects and random effects. Examine marker-pair residual plots and correlations, and look for systematic changes by visit, time, or subject characteristics. Also check whether residual association persists across distinct visits; if it does, the assumption of no cross-visit residual covariance may be questionable. These checks are exploratory: fitted random effects are estimated, so conditional residuals can be shrunken and their raw sample covariance need not be an unbiased estimate of \(\Sigma_{\epsilon}\).

Warning signs include an estimated covariance matrix near singularity, correlations close to their allowable boundaries, unstable estimates across reasonable starting values, and implausible changes in estimates when a few subjects are removed. Near-singularity can arise when the data provide little information to distinguish residual covariance from random-effect covariance. The convergence and stability diagnostics used for mixed models remain relevant, but convergence alone does not establish that a covariance structure is adequate.

The choice is a balance. Independent errors are efficient and interpretable when cross-marker residual association is negligible after accounting for random effects. An unstructured matrix avoids strong equality constraints but can require substantial information, particularly as \(p\) grows. A parsimonious structure can stabilize estimation, but a poor restriction may leave systematic residual patterns or distort other model estimates. Compare scientifically plausible alternatives using suitable model assessment, and consider whether conclusions are sensitive to the choice. A covariance pattern describes conditional dependence; it does not establish a biological mechanism or a causal relation.