Within-Subject Correlation in the Population Fisher Matrix

Why the Off-Diagonal Entries Matter

Repeated observations on one subject share sources of variation. A subject-specific random effect can move all of that subject’s measurements together, while correlated residual errors can make nearby observations more alike than distant ones. These dependencies appear as off-diagonal entries in the covariance matrix of the subject’s observation vector. They affect how much independent information the observations contain, and which population features they inform.

The effect is not always a simple loss of information. Positive correlation can make repeated measurements less informative about a common population level, because they tend to move together. But it can make a within-subject contrast more precise when the shared component cancels in that contrast. The impact depends on the parameter being estimated and on how the design maps parameters into observations.

Constructing the Approximate Observation Covariance

For subject \(i\), let \(y_i\) be the vector of \(m_i\) measurements at the planned observation times. Write the model as \(y_i=f_i(\theta,\eta_i)+\epsilon_i\), where \(\theta\) denotes population parameters, \(\eta_i\) is a mean-zero vector of subject-specific random effects with covariance \(\Omega\), and \(\epsilon_i\) is a mean-zero residual vector with covariance \(R_i\). Assume random effects and residual errors are independent, and that different subjects are independent.

A first-order approximation expands the individual prediction around the mean random effect, \(\eta_i=0\). Define \(Z_i\) as the matrix of derivatives of the predicted observations with respect to the random effects, evaluated at that point. The resulting approximate model is \(y_i\approx f_i(\theta,0)+Z_i\eta_i+\epsilon_i\). Its observation covariance is

The first term is covariance induced by random effects; the second is residual covariance. In particular, for observations \(j\) and \(k\), the approximate covariance is \(z_{ij}\Omega z_{ik}^\mathsf{T}+(R_i)_{jk}\), where \(z_{ij}\) and \(z_{ik}\) are the corresponding rows of \(Z_i\). Their correlation is this covariance divided by the product of their marginal standard deviations. A covariance matrix and a correlation matrix are therefore related but not interchangeable.

This approximation requires a differentiable individual prediction function near \(\eta_i=0\), finite random-effect and residual covariance matrices, and a valid joint covariance: \(V_i\) must be symmetric and positive definite for the inverse-based Fisher information below. It is a local approximation; strong nonlinear effects of random effects can make it inaccurate. Residual independence is not required: it is represented by diagonal \(R_i\) only when that assumption is appropriate.

How Covariance Enters Population Information

Under the approximate multivariate normal model \(y_i\sim N(\mu_i,V_i)\), let \(\mu_i\) be the approximate marginal mean and let \(\psi\) collect the population parameters of interest. Write \(\mu_{i,a}=\partial\mu_i/\partial\psi_a\) and \(V_{i,a}=\partial V_i/\partial\psi_a\). For a differentiable mean and covariance and nonsingular \(V_i\), the expected Fisher information contributed by subject \(i\) has entries

The first term is information carried by changes in the mean, weighted by the inverse observation covariance. The second term is information carried by changes in the covariance itself. If a parameter affects only the mean, its covariance derivative is zero and the second term disappears. If a parameter affects both, both terms contribute. For independent subjects, their information matrices add.

The inverse covariance weights directions in the observation vector differently. A pattern that varies strongly under \(V_i\) receives less weight; a pattern with little variability receives more. Thus, off-diagonal entries matter even if every observation has the same marginal variance: they change the variability of averages, contrasts, and other patterns in the data.

A Short Justification of the Fisher Formula

For one subject, the multivariate normal log likelihood, omitting constants, is \(-\tfrac12\log|V_i|-\tfrac12(y_i-\mu_i)^\mathsf{T}V_i^{-1}(y_i-\mu_i)\). Differentiating with respect to \(\psi_a\) gives a mean score \(\mu_{i,a}^\mathsf{T}V_i^{-1}(y_i-\mu_i)\) and a covariance score

Under the model, \(y_i-\mu_i\) has mean zero and covariance \(V_i\). The expected product of the mean and covariance scores is zero because it involves centered third moments of a multivariate normal vector. The expected product of two mean scores is \(\mu_{i,a}^\mathsf{T}V_i^{-1}\mu_{i,b}\). For the covariance scores, the centered quadratic-form identity gives one half of \(\operatorname{tr}(V_i^{-1}V_{i,a}V_i^{-1}V_{i,b})\). Adding these expected score products yields the stated information. The argument depends on the approximate normal model and does not assert that the original nonlinear mixed-effects model is exactly normal after marginalization.

Worked Example: Correlation and Information About a Common Mean

Worked Example: Building a Random-Intercept-and-Slope Covariance

Correlation Can Help a Contrast

Diagnostics, Alternatives, and Limitations

Before using a proposed covariance matrix, check that it is symmetric, has positive diagonal entries, and is positive definite for calculations that require \(V_i^{-1}\). Check the model’s implied correlations against the measurement process: random intercepts tend to induce shared covariance, random slopes can make covariance depend on the observation times, and residual correlation can decay with lag or follow another specified structure. A mathematically valid matrix can still be a poor description of the data-generating process.

A common failure is to use a diagonal \(R_i\) simply because each measurement has a residual variance. That assumes residual errors are uncorrelated across the subject’s observations. If this is false, the diagonal model generally gives the wrong weighting of averages and contrasts, and can misstate information for population parameters. Compare plausible residual structures and inspect within-subject residual patterns; sensitivity of the information calculation to those choices is itself useful design information.

The first-order covariance approach is a practical local approximation, not a universal replacement for the nonlinear model. It may be unreliable when random effects are large, the prediction function is strongly curved over their distribution, or the residual distribution is far from the assumed normal form. A conditional analysis that treats random effects as known targets a different information calculation; it should not be substituted for marginal population information without stating the change in target. More elaborate marginal calculations may be needed when the first-order approximation is inadequate.

• Mistake: “Each subject has two observations, so there are twice as many independent pieces of information.”
  Correction: Repeated measurements are not independent when \(V_i\) has off-diagonal entries. Their contribution depends on the covariance and on the parameter-sensitive pattern of the observations.
• Mistake: “Positive correlation always lowers information.”
  Correction: It often lowers information about a common level at fixed marginal variance, but it can increase information about a contrast if shared variation cancels.
• Mistake: “The random-effect covariance alone is the observation covariance.”
  Correction: Add the residual covariance \(R_i\); both components contribute to the joint distribution of observations.